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Executive Summary 


Introduction 

This report presents Year One (2010-11) school level achievement and attendance outcomes and case 
study findings from fall 2011 that focused on school leadership and instruction. 

Thirteen schools were included in the first year of Philadelphia’s Renaissance Schools Initiative 
(2010-11). Of these schools: 


Four K-8 schools were District- 

Seven K-8 schools were operated 

Two high schools were District- 

operated as Promise Academies 

U "1 

by four charter school providers 

run Promise Academies 


The analyses provide the most rigorous evaluation available regarding the impact of the reform model. 

First -year results are positive. However, because the outcomes analyses were conducted using data 
from only the first year of the Initiative, it is too early to determine whether the preliminary results 
summarized here will be sustained over time. 

Student Outcomes: Renaissance Schools vs. Comparison Schools 

K-8 Schools (11 Schools) 

Student achievement and attendance at 11 K-8 Renaissance Schools during the first year of 
implementation were compared to a group of 72 K-8 schools with similar School Performance Indices 
(SPIs). Performance was tracked five years prior to the beginning of the Initiative (2006-10); and after 
one year of the Renaissance Schools Initiative (2010-11). 


Finding 1 

Student achievement in K-8 Renaissance Schools improved to a significantly greater degree than did 
achievement in similar schools not included in the Initiative. Specifically, Renaissance school student 
achievement gains significantly outpaced those of Comparison Schools on the following measures: 

• Math and Reading PSSA scale scores 

• Percentage of Students Proficient or Above in Math and Reading 

• Percentage of Students Below Basic in Math and Reading 
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Finding 2 

Student attendance at Renaissance Schools increased significantly more than did attendance at 
Comparison Schools. 


The school-level effect sizes associated with participation in the Renaissance Schools 
Initiative for both student achievement and attendance ranged from 1.0 to 2.25, which 
are substantially greater than is typically seen in educational research. 

High Schools (2 Schools) 

Due to the small number of schools, meaningful significance tests of differences between Renaissance 
High Schools and Comparison High Schools were not possible. Findings from descriptive analyses are 
as follows: 


Finding 3 


Descriptive analysis of student achievement and attendance reveal no observable changes at 
Renaissance High Schools in Year One of the Renaissance Schools Initiative, while the Comparison 
High Schools have continued to improve slightly over the six-year study period. 


Student Outcomes: Promise Academies vs. Charter-Managed Schools 

The 11 K -8 Renaissance middle schools were operated by both the District and Charter providers. We 
conducted analyses to estimate differences among the Renaissance Charters, Promise Academies, and 
the Comparison Schools. Results of these analyses are as follows: 


Finding 4 

There was no statistically significant difference in either student achievement or attendance between 
the K-8 Promise Academies and the Charter-managed Renaissance Schools. Both sets of schools 
significantly out-performed the Comparison Schools in terms of increases in student achievement 
and attendance. 


Student Outcomes: Comparisons by Individual Provider 

The 11 K -8 Renaissance middle schools in Year One were operated by five providers: District-run 
Promise Academies (four schools); Mastery (three schools); Universal (two schools); ASPIRA (one 
school); and Young Scholars Academy (one school). Due to the small number of schools overseen by 
each provider, it was not possible to assess the significance of observed differences among the five 
Renaissance School providers. Results of descriptive analyses are as follows: 
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Finding 5 

Descriptive comparisons of Renaissance Schools reveal roughly equivalent performance along 
student achievement and attendance measures across all operators. 


It is important to note that the schools chosen to participate in the Renaissance initiative were among 
the lowest-performing schools in the District at the start of the reform effort. Even with the significant 
gains in Year One of the Initiative, they remain among the lowest-performing schools in the District. 


Year Two: Promising Practices 

This report also highlights promising District policies and school practices that could impact future 
turnaround efforts in Philadelphia or in other locales. These include the following: 

1) The Promise Academies received extra resources and attention from the District, which placed 
them in the spotlight and generated greater public interest in their progress. 

2 ) The teachers’ union supported the Renaissance Schools Initiative by signing a collective 
bargaining agreement that was consistent with the principles of the reform effort. 

3) Principals and teachers felt part of “something big.” 

4) Principals built their own teams of teachers through site selection. 

5) Principals built systems that promoted and reinforced teacher learning and growth. 

6 ) Data and student work were used to assess learning and make instructional decisions. 

7 ) Principals and teachers exercised professional judgment to adapt the curriculum, within the 
parameters of the Promise Academy Way. 


Year Two: Emerging Challenges 

The context in which Year Two (2011-12) of the Renaissance Schools Initiative unfolded changed 
markedly from that of the first year of the reform. The Renaissance Schools Initiative was conceived and 
implemented at a time when the District had significantly more financial resources, but 2011 brought 
drastic cuts in state funding that resulted in a substantial reduction in Central Office staffing, deep 
teacher and staff layoffs, and widespread program and budget cuts. Additionally, there was an 
unprecedented amount of turnover, including top leadership positions and the departure of former 
CEO Arlene Ackerman. 
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During 2011, Renaissance Schools, particularly the District-run Promise Academies, were challenged 
by: 


1) A late hiring window, with little time for orientation. 

2) Significant staff and programmatic reductions as a result of District cut-backs. 

3 ) High levels of teacher turnover. 

4 ) Over-use of corrective reading and math programs intended to increase student achievement. 

5 ) Over-reliance on direct instruction, particularly around prescribed curricula. 


Implications and Next Steps 

The Year One outcomes for schools in the Renaissance Schools Initiative suggest that something 
positive is happening in the first cohort of Promise Academies and Renaissance Charters. Year One of 
the Initiative saw significant gains in the rate of student achievement and attendance; and Year Two 
case studies point to an emerging set of promising practices that are likely to be related to the success of 
these schools. But a set of significant challenges also emerged in Year Two. These important contextual 
factors will require careful monitoring as the Initiative continues. 

Ongoing research should address the following questions: 

1) Does the rate of growth in student achievement at Renaissance Schools continue in subsequent 
years? 

2) Is the model scalable— that is, as it expands to include more schools, can the successes of the 
Year One cohort be replicated? What level of resources or supports is necessary to achieve this 
goal? 

3) What is the cost of implementing successful Renaissance schools using the Promise Academy 
model? The Renaissance charter model? In K-8 schools? In high schools? 

4) Do differences between providers emerge over time? 

5) Does the success of the model differ by type of school (K-8, high school) or student population 
(special education, ELL, low-income, high-performing, low-performing) 

6) At both Promise Academies and Renaissance Charters, what replicable policies and practices are 
contributing to increased student achievement? 

7) What is the collateral effect for comparable District schools not selected for the Renaissance 
Schools Initiative? 


IV 
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Introduction 

In May 2010, the Accountability Review Council (ARC) commissioned Research for Action (RFA) to 
evaluate Philadelphia’s Renaissance Schools Initiative. The Renaissance Schools Initiative was the 
signature reform of Dr. Arlene Ackerman, Superintendent of the School District of Philadelphia (the 
District) from 2008 through 2011; the reform has continued since her departure. Implemented in 2010 
at a time of greater District resources, the Renaissance Schools Initiative targets a select number of 
persistently low performing schools by providing additional resources, increased attention, and 
strategies designed to spur dramatic increases in student performance in a few years. 


In its first year, the Renaissance Schools Initiative consisted of two models: 


Six Promise Academies 

Seven Renaissance Charter Schools 

Remained under District management and 
underwent reforms similar to “turnarounds” as 
defined by the U.S. Department of Education. 1 
Two principals who had been at their schools for 
only one year were permitted to stay and all the 
other principals and at least half the teaching staff 
at every school were replaced. In addition, learning 
time was extended, and schools received additional 
funding and support staff. 

Were matched with charter managers, similar to the 
federal “restart” model, but remained neighborhood 
schools. They operated with relative autonomy 
from the District, which allowed for variation in the 
reforms implemented at the schools. Renaissance 
Charters received some District oversight and 
support, and regularly reported to the District on 
student outcomes. 


In September 2010, the first 13 Renaissance schools opened their doors. 2 In the first year of the 
Initiative the schools faced three overarching challenges: 

1) The process of assigning schools to models began in March 2010; matching charter providers to 
schools was not completed until May 2010, which resulted in a rushed timeline for staffing 
schools and preparing for school start up. 


1 U.S. Department of Education (2010). Guidance on School Improvement Grants Under Section 1003(g) of the Elementary 
and Secondary Education Act of 1965. Washington, DC: Author. 

2 The 13 schools included elementary and K-8 schools, two middle schools, and two high schools. Both high schools were 
Promise Academies. 
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2) In some charter-managed schools, as well as the Promise Academies, the guidelines for behavior 
and new curriculum were being implemented throughout the fall, leaving school staff feeling as 
if the District was “inventing as you go.” 

3) Central Office staff had to address the many new tasks associated with the reform with limited 
District capacity, which was further exacerbated by the rushed timeline and the reality that 
reforms were being continuously invented and implemented^ Additionally, the Central Office 
departments responsible for supporting these new schools went through multiple leadership 
changes throughout the first year. 

In Year Two (September 2011), the first 13 Renaissance Schools re-opened, along with 10 new 
Renaissance Schools added to the Initiative. The previous summer had seen the turnover of the 
superintendent, and significant cuts in the District’s budget, leading to substantial Central Office staff 
reductions and District-wide teacher layoffs. A timeline on page 3 chronicles these and other major 
events since the District first launched the Renaissance Schools Initiative. The impact of these changes 
on the reform effort is significant, and still unfolding. 

Th is Report: Renaissance Schools Initiative at 18 Months 

The ARC originally commissioned RFA to conduct a one-year (May 2010-May 2011) mixed-methods 
study of the start up of the Renaissance Schools Initiative. In May 2011, the ARC extended RFA’s 
contract to December 2012, specifically to assess the impact of the Initiative through an examination of 
two years of student outcomes, as well as to contextualize these analyses with a set of case studies 
focused on implementation and factors that supported or inhibited positive change. 

This report presents Year One outcomes and Year Two case study findings focused on school leadership 
and instruction (data from Year One was unavailable at the time of our May 2011 report). A future 
report will present Year Two outcomes and case studies examining school climate and community 
ownership. 3 4 Each section of this interim report provides guiding research questions and the data 
sources used to address them. Appendices A-D provide supplemental findings, along with detailed 
quantitative and qualitative methodologies. The sections that follow this introduction are: 

• Year One: Promising Outcomes 

• Year Two: A Closer Look at Leadership and Instruction in Two Promise Academies 

• Conclusion 


3 Gold, E., Good, D., Roberson-Kraft, C., & Callahan, M. K. (2011). Philadelphia’s Renaissance Schools: A report on start up 
and early implementation. Philadelphia: Research for Action. 

4 Although community ownership through the School Advisory Councils was not a major focus of our fall 2011 research, we 
include an update on development in Appendix A. 


2 


Figure. 1 Timeline 


April 2009 - Ackerman 
announces Imagine 2014, 
five-year strategic plan 


March 2010 - School Advisory 
Councils (SACs) form at Ren. Schools 

March 2010-6 of the Ren. 
Schools are designated Promise 
Academies 


April 

2009 


a 


ip 


esi 


Jan 2010 - District 
and Philadelphia 
Federation of Teachers 
(PFT) sign collective 
bargaining agreement, 
which includes 
concessions that allow 
for the Renaissance 
Schools Initiative to 
move forward 


June 2010 - After controversy, West 
Philadelphia High School turnaround is 
postponed, reducing the number of Ren. 
Schools to 13 


May - Aug 201 0 - Ren. Schools 
hire teaching staff; hiring freeze for 
non-Ren. Schools 


May 

2010 

start 
of our 
research 


May 2011 - After controversy, SAC votes 
and District announces that MLK High School 
will become a Promise Academy instead of 
being run by a charter manager 

May 2011 - School Reform Commission 
(SRC) passes District budget, which is 
significantly reduced from prior year 


Jan 2011 - District 
identifies second cohort 
of 18 Ren. Schools (10 
will be run by the District, 
8 by charter managers) 


Feb 2011 - SACs form 
at second-cohort schools 


Dec 2011 - Schools districtwide face 
additional budget cuts including 2% from 
Promise Academy operating budgets; 
the Promise Academy office is closed 


June 2011 - PFT sues 
the District for exempting 
Promise Academies from 
districtwide teacher layoffs; 
all layoffs are put on hold 


Sept 6, 2011 - School 
begins; 13 Ren. Schools enter 
their second year; 10 enter 
their first 




lacrt IOIOI 


n 


May 2010 - 7 of the Ren. Schools 
match with 4 charter managers 


Indicates 8 month 
time lapse 

May 2009 -Jan 2010 


March 2011 -6 of the second- 
cohort schools match with 3 charter 
managers; another 2 will be managed 
by Universal Companies under a 
Promise Neighborhood grant 


Sept 2010 - School year begins 
for the first cohort of 13 Ren. 
Schools (6 Promise Academies, 7 
charter-managed) 


August 3, 2011 - In 
efforts to close the budget 
gap, SRC approves cutting 
the number of second- 
cohort Promise Academies 
to 3, and makes cuts to the 
Promise Academy model 


August 16, 2011 - PFT 

and District settle out of 
court; Promise Academy 
teachers will not be 
exempted from seniority- 
based layoffs, resulting in 
25-40% turnover at these 
schools. Promise Academies 
scramble to fill their 


August 22, 2011 - 

Ackerman resigns; Leroy 
Nunery is appointed acting 
superintendent 

Jan 2012 - Renaissance Request for 
Proposals from charter managers is 
released for Year Three 

Jan 2012 - SRC hires Knudsen as acting 
superintendent & Chief Recovery Officer 


Jan 

2012 


District Leadership 


Superintendent 


Leroy Nunery II, 


Tom Knudsen, Acting Supt. & 


1 “ 1 — 1 '1 


I 




















August 

2011 




Dec 

2011 




April 

2009 


Jan 

2012 


Promise Academy Oversight 


Joel Boyd, Assistant Superintendent Oversight given to 


1 I 1 — 




May 














2011 

L _2 _ Jl 

Dec 

2011 


April 

2009 


Jan 

2012 


Charter-managed Ren. Schools Oversight 



Diane Castelbuono. Deputy of Renaissance Schools 


John Frangipani, Assistant 
Supt. for Renaissance 
Charters & CTE Programs 


Thomas Darden, Deputy 
Chief - Strategic Programs 


April 

2009 


Dec 

2011 


Jan 

2012 
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Year One: Student Outcomes 

During Year One of the Renaissance Schools Initiative, RFA conducted interviews, focus groups, and 
observations in all 13 first-cohort schools. While we found variation across schools in their 
implementation of the Initiative, the majority of schools showed early improvements in attendance and 
reported calmer, safer school climates. All 13 schools had School Advisory Councils (SACs), albeit with 
varying degrees of functionality. Despite a rushed timeline to open school on time, all 13 schools had 
site-selected their teaching staff and were - to varying extents - establishing systems to support and 
monitor instruction, s 

At the time of our May 2011 report, end-of-year data on outcomes were not yet available, and therefore 
we could not assess academic achievement in the Renaissance Schools. In the months since, RFA has 
obtained District and charter data from the first year of the Initiative (the 2010-11 school year), and 
performed analyses guided by the following research questions: * * * 6 

1) How do changes in the Renaissance Schools compare to those of Comparison Schools in Year 
One of the Renaissance Schools Initiative? 

• Student enrollment 

• Student retention 

• Student demographics 

• Student: teacher ratios 

2) How do changes in the Year One performance of Renaissance Schools compare to those of 
similar schools along the following outcomes? 

• Performance on the PSSA Math and Reading Assessments (PSSA Scores and PSSA 
Performance Levels) 

• School Average Daily Attendance 

This section presents analyses that compare the 13 first-cohort Renaissance schools to a comparison 
group of similar District schools. The comparison group selection process is detailed in the next section. 
We begin with a brief overview of our research methods (more detail is available in Appendix C), 
followed by descriptive comparisons of student enrollment, mobility, demographics, and student: 
teacher ratios. Finally, we report on school outcomes through impact analyses of academic performance 
and school attendance. 


s Here and throughout this section, we make reference to research presented in our earlier report: Gold, E., Good, D., 

Roberston-Kraft, C., & Callahan, M. K. (2011). Philadelphia’s Renaissance Schools: A report on start up and early 

implementation. Philadelphia: Research for Action. 

6 The original proposal also called for analyses of school climate indicators, along with teacher demographics and turnover. 
These indicators were either found to be unreliable indicators that are not systematically collected across the District, such as 
suspensions and violent incidents, or the District has not yet provided the data in response to our data request, such as teacher 
demographics, experience and mobility. 
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Research Methods 


Data and Methods 

The District provided six years of school-level records for every Year One Promise Academy and 
Comparison School in RFA’s study. In addition, the District provided five years of school-level records 
prior to the beginning of the Renaissance Schools Initiative (years spanning 2005-06 to 2009-10) for 
every Renaissance charter school. Renaissance Charter providers supplied school-level records for the 
2010-11 academic year. 

The descriptive analyses present basic comparisons among three groups - Promise Academies, 
Renaissance Charters, and Comparison Schools - along the following indicators: 

1) School enrollment 

2) Student transfers 

3) Student demographics 

4) Student:teacher ratio 

These analyses establish the overall equivalence of the Renaissance Schools and the Comparison 
Schools, while highlighting some key descriptive differences among these groups. 

The impact analyses assess the school-level impact of participation in the Renaissance Schools 
Initiative using a series of multi-level regression models. 7 These models were designed to estimate the 
school-level change of six different outcome measures of student achievement and attendance, while 
controlling for change over time and for school-level indicators commonly known to exert an influence 
on student achievement and attendance. These analyses were specifically designed to assess the 
difference between the Renaissance Schools and the Comparison Schools for each outcome measure. 
Detailed tables of school-level performance on each outcome measure are included in Appendix B. 

Comparison Schools 

The Comparison Schools for the impact analyses in this section are comprised of 72 K-8 and 19 high 
schools with a School Performance Index (SPI) 7 8 9 score of 7-10 at the end of the 2009-10 academic year. 
The SPI rates every school in the District on a scale of 1 to 10, where 1 indicates a school in the top 
decile, and 10 in the lowest. The index is based on Student Progress (growth on PSSA); Student 
Achievement (PSSA); Postsecondary Readiness; and Satisfaction and Engagement of Students, Parents 
and Teachers. In consultation with District research staff, the SPI measure was used as the standard for 
selecting the comparison group of schools since the SPI was the metric used in the identification of 
Renaissance Schools.? The comparison group chosen for these analyses shared characteristics most 
similar to those of the Renaissance Schools (see Appendix B for historical comparisons between the 
Renaissance Schools and the Comparison Schools). 


7 For a detailed account of the modeling used for these analyses and outputs from these models please see Appendix C. 

8 See: http://webgui.phila.k12. pa.us/offices/a/accountability/school-performance-index-spi 

9 Renaissance Schools were identified based on being in the pool of schools with an SPI of 10, the lowest rating. 
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Setting the Stage: Renaissance School Profiles 

School Enrollments and Populations 

In November 2010, data provided by the District and charter providers showed significant increases in 
student enrollment at Renaissance charter schools, but not at the District-run Promise Academies. 
Figure 2 shows that at the end of the 2010-11 school year, overall enrollments at Renaissance Charters 
increased, while enrollments at Promise Academies and the Comparison Schools continued on a 
downward trajectory. 

Interviews in fall 2010, at the outset of the initiative, suggested that when the school buildings were 
transitioned to charter-run schools - a process which involved significant parent and community 
member input - they attracted increased attention; Promise Academies, however, did not receive the 
same level of visibility. The additional attention received by the Renaissance Charters may help to 
explain their increased enrollments in the first year of the Renaissance Schools Initiative. 

As seen in Figure 3, on average, a lower percentage of students enrolled in Renaissance Charters 
transferred out of their schools in the first year of the Initiative than transferred out of Promise 
Academies or the Comparison Schools. These differences are likely magnified due to the sharp increase 
in enrollment at the Renaissance Charters throughout the course of the 2010-11 academic year. 
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Figure 2. School Enrollment 


Figure 3. Voluntary Student T ransfers 10 


School Enrollments 

(End of Year Enrollment) 



Comparison Schools Promise Academies 

Renaissance Charters 

n = 13 Renaissance Schools; 90 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Student Voluntary T ransfers 

(As a Percentage of Total School Enrollment) 



Comparison Schools Promise Academies 

1 Renaissance Charters 

n = 13 Renaissance Schools; 90 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006-10 enrollments, at all schools were steadily 
declining. 

After one year of the Renaissance Schools Initiative (2011): 

• The newly opened Renaissance Charters increased 
enrollments at their schools, while the Promise Academy and 
Comparison School enrollments continued to decline. 


Prior to the Renaissance Schools Initiative: 

• From 2006-10, the percentage of students transferring out of 
future Promise Academies and the Comparison Schools was 
roughly equivalent, with the future Renaissance Charters 
having slightly higher transfer rates. 

After one year of the Renaissance Schools Initiative (2011): 

• A smaller percentage of students at Renaissance Charters 
transferred out of their schools than from either the Promise 
Academies or the Comparison Schools. 


10 A percentage of ‘voluntary transfers’ was calculated as the number of voluntary transfers divided by (total end of year enrollment + total voluntary transfers + total 
disciplinary transfers). 
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Figure 4 provides an overview of student demographics in the Promise Academies, Renaissance Charters, and Comparison Schools. The 
higher percentage of English Language Learners (ELLs) at the Promise Academies reflects the concentration of ELL students in two of the 
four Promise Academies. 


Figure 5 reveals that in Year One, the student:teacher ratio at Renaissance Charters slightly increased from the previous year and was 
roughly five students per teacher higher than at Promise Academies. The student :teacher ratio is derived from the total number of 
students in a building to the total number of full-time equivalent teachers in a building. This does not equate to class size; 
classroom teachers represent only a portion of full-time equivalent teachers. 


Figure 4. Student Demographics 


Student Demographics 

(Academic Year: 201 0-2011) 



n = 13 Renaissance Schools; 91 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 






Schools in the study served roughly similar student 
populations in 2010-11. 

o These student populations have remained consistent 
throughout the study period: 2005-06 through 2010- 
11 (Gold et al., 2011) 

Renaissance Charters served slightly fewer Special Education 
and ELL students than either the Promise Academies or 
Comparison Schools in 2010-11. 


Figure 5. StudentiTeacher Ratio 


Student : Teacher Ratio 

Full Time Equivalent Teachers 



Comparison Schools Promise Academies 

Renaissance Charters 

n = 13 Renaissance Schools; 90 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• Student:teacher ratios had been declining in all schools from 
2007 through 2010. 

After one year of the Renaissance Schools Initiative (2011): 

• Promise Academies had the lowest student:teacher ratio, just 
below 10:1, while Renaissance Charters’ average student: 
teacher ratio increased to just under 15:1. 
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Assessing Impact: School Attendance and Academic Performance 11 

The Renaissance Schools Initiative was conceived as a school turnaround effort designed to generate 
drastic student achievement gains in two to three years. Our analysis compares the historical 
performance of the first cohort of Renaissance Schools to Comparison Schools, and includes 
performance outcomes for five years prior to the Initiative plus the first year of the Initiative. The 
results presented in the following pages show that: 


In Year One, attendance and academic performance at Renaissance Schools improved significantly 
more than at Comparison Schools. This improvement was true for both the Promise Academies 
and the Renaissance Charters; no statistically significant differences were found between these two 
groups. 


Regression Model Indicators 

Multi-level regression models were developed for our outcome measures to assess the impact of 
participation in the Renaissance Schools Initiative. (See outcome measures listed in Table l, below.) 
The Renaissance Schools Initiative is included in each model as the primary “treatment,” along with 
school-level control measures commonly associated with student achievement and attendance. A full 
description of the formal modeling strategy, along with the outputs for each model, is outlined in 
Appendix C. 

In addition to the seven models run to compare the performance of all Renaissance Schools with the 
Comparison Schools, seven additional models were run to assess whether there were statistically 
significant differences along each outcome between Renaissance Charters and Promise Academies. 
These models did not reveal any significant differences between Renaissance Charters and Promise 
Academies along any of the outcome measures considered. The results of these models are presented in 
Appendix C. 


11 High Schools were excluded from the impact analyses for the following reasons: 1) With only two Promise high schools, this 
did not represent a large enough ‘treatment group’ to perform a separate high school impact analysis for the Initiative. 2) The 
general non-equivalence of K-8 schools and high schools makes combining the performance of 11 th graders with that of 3 rd - 8 th 
graders misleading since traditionally high schools respond quite differently to reform efforts than K-8 schools. High schools 
are more complex institutions, and have historically had a much worse track record than K-8s in school turnaround initiatives, 
(e.g., de la Torre, M., Allensworth, E., Jagesic, S., Sebastian, J., Salmnowicz, M., Meyers, C., & Gerdeman, R. D. Turning 
around low-performing schools in Chicago. Chicago: University of Chicago Consortium on Chicago School Research.) 3) With 
so few schools in the Initiative, including high schools in the ‘treatment group’ would have likely exerted a negative influence 
on the overall performance of the Renaissance Schools group as a whole, potentially masking impact at the K-8 level. 4) There 
were no Renaissance Charter high schools in Year One, and therefore no point of comparison between the two Renaissance 
models. 
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Table 1. Regression Model Indicators 


School Level Outcome Measures 

Treatment 

Control Measures 

Student Achievement 

1) Average PSSA Scaled Scores - 
Math 

2) Average PSSA Scaled Scores - 
Reading 

3) % of Students Proficient or Above 

- Math 

4) % of Students Proficient or Above 

- Reading 

5) % of Students Below Basic - Math 

6) % of Student Below Basic - 
Reading 

Attendance 

7) School Average Daily Attendance 

Participation in 
the Renaissance 
Schools Initiative 

Time 

• Pre-Intervention 
(2005-06 -> 2009-10) 

Comparison Group 

• Elementary and Middle schools with 
an SPI of 7-10 in the 2009-10 
academic year 

Grade Level 

• 3 rd grade as the comparison group 
School Level Controls 

• Elementary v. Middle School 

• Socio-demographics 
0 % ELL, Special Ed., 

Free/Reduced Lunch 

• Enrollment 

• Student Retention 12 

• Average Teacher Years at current 
school 

• Student: Teacher Ratio 13 


Regression Model Findings 

Table 2 presents estimates from the seven different regression models displaying the statistical 
significance of differences between Renaissance Schools and the Comparison Schools for each outcome 
measure, controlling for all of the other factors listed in Table l, above. The columns in Table 2 present 
for each school level outcome: l) the estimated amount of the initial difference between Renaissance 
Schools and the Comparison Schools in 2005-06; 2) the estimated amount of the difference between 
Renaissance Schools and the Comparison Schools’ rate of growth from 2005-06 through 2010-11; and 
3) the estimated amount of difference in the overall amount of change between Renaissance Schools 
and Comparison Schools in Year One of the Renaissance Schools Initiative (2010-11). 


12 “Student Retention” was calculated as the percentage of students who attended a single school in consecutive years, i.e. for 
the 2010-11 school year a school’s ‘student retention’ would be calculated at the percentage of students in 2010-11 who also 
attended the same school in 2009-10. 

'3 Data on “Teachers” provided by the District and charter school providers represent the number of full-time-equivalent 
teachers at each school for each year. Full-time equivalent teachers include Bilingual Teachers, Art Teachers, Special Education 
Classroom Teachers, Title I “Like” Staff, Guidance Counselors, Teacher Aides (instructional & certified), Librarians, Music 
Teachers, Physical Education Teachers, Psychologists, Social Workers, Speech Therapists, Department Heads, and School 
Based Instructional Staff. 
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Table 2. Student Outcomes: Renaissance Schools vs. Comparison Schools 14 


Outcome 

Initial Level 
Difference 5 

( 2005 - 06 ) 

Growth Rate 
Difference 16 

Post-Intervention 
Difference 7 

( 2010 - 11 ) 

Math Scale Score 

-46.3** 

-2.5 

82.5** 

Math % Proficient 
or Above 

-8.2** 

-1.1 

17.9** 

Math % Below Basic 

8.8** 

0.7 

-18.1** 

Reading Scale Score 

-36.6** 

-2.3 

64.2** 

Reading % Proficient 
or Above 

-7.2** 

-0.9 

11.1** 

Reading % Below Basic 

8.1** 

0.5 

-16.7** 

Attendance Rates 

-0.97 

-0.02 

1.49* 


* Indicates that differences are statistically significant at p<.05 
** Indicates that differences are statistically significant at p<.oi 


As an example of how to read Table 2, consider Math percent of proficient or above: 

As expected given the selection criteria, in 2005-06 the Renaissance Schools performed significantly 
worse on average than the Comparison Schools, with roughly 8% fewer students scoring proficient or 
above on the Math PSSA. There is no statistical difference between the future Renaissance Schools’ 
growth rate and the Comparison Schools’ growth rate. However, in Year One of the Initiative, the 
Renaissance Schools increased the percentage of their students scoring proficient or above on the 
Math PSSA on average by almost 18 percentage points more than the Comparison Schools. 


14 It is important to note that the findings presented represent only a single year of observed performance of Renaissance 
Schools in 2010-11. With only one year of ‘post-implementation’ results it is not possible to estimate ‘post-implementation 
growth’ along each of the outcomes. The observed levels of each outcome simply point to a single year shift in the performance 
of the Renaissance Schools and cannot be interpreted as shifts in the rate of growth at these schools. 

Initial Level Difference represents the difference between the Renaissance School Averages and the Comparison School 
Averages after controlling for the following cohort and school level factors: grade level, school grade configuration, school 
enrollment, student socio-demographics (percent Minority, ELL, Special Ed., Free Lunch), student retention, average teacher 
years at school, and student : teacher ratio. 

16 Growth Rate Difference represents the difference between the future Renaissance Schools’ average rate of growth and the 
Comparison School average rate of growth for each outcome from 2005-06 through 2010-11, controlling for all the other 
indicators included in each model. 

v Post-Intervention Difference represents the difference between the Renaissance Schools’ average change for each outcome 
and the Comparison Schools’ average change for each outcome in Year One of the Renaissance Initiative, controlling for all the 
other indicators included in each model. 
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The summary findings in Table 2 reveal: 


Schools that would become 
Renaissance Schools 
performed at a significantly 
lower level than Comparison 
Schools along each outcome in 
2005 - 06 . 

Schools that would become 
Renaissance Schools and 
Comparison Schools experienced 
roughly equivalent growth rates 
along each outcome from 2005-06 
through 2010-11. 

Renaissance Schools experienced 
significantly more improvement 
than the Comparison Schools 
along each outcome in Year One 
of the Renaissance Schools 
Initiative. 



Academic Achievement and Attendance Outcome T rends Over Time 

Figures 6 through 12 present a set of comparative trend lines for the observed performance of school- 
level academic achievement and attendance outcomes for the Renaissance Schools and the Comparison 
Schools. 


The analyses presented here suggest that the Renaissance Schools Initiative exerted a positive 
influence on participant schools in Year One, narrowing the gap between Renaissance Schools and 
the Comparison Schools along every outcome. 
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Figure 6. PSSA Math Scores 18 


Figure 7. PSSA Reading Scores 


Student Achievement 

PSSA Math: Scale Scores (Grades 3-8) 



Renaissance Schools Comparison Schools 

n = 1 1 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006-10, the Comparison Schools’ average math scale 
scores were higher than the future Renaissance Schools’ 
scores at a constant margin. 

After one year of the Renaissance Schools Initiative (2011): 

• The Renaissance Schools’ performance on the Math 
PSSA was roughly equivalent to the Comparison 
Schools. 


Student Achievement 
PSSA Reading: Scale Scores (Grades 3-8) 



Renaissance Schools Comparison Schools 

n = 1 1 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006-10, the Comparison Schools’ average reading 
scale scores were higher than the future Renaissance Schools’ 
scores at a constant margin. 

After one year of the Renaissance Schools Initiative (2011): 

• The Renaissance Schools’ performance on the 
Reading PSSA was roughly equivalent to the 
Comparison Schools. 


18 Scale Scores are the unit of measurement for the PSSA Math and Reading assessments. The range of possible scores increases with each grade level making the overall 
school average scale scores a difficult number to interpret. The key findings in Figures 6 and 7 are the upward trends of average scale scores and the significant 
improvements observed in Year One of the Renaissance Schools Initiative. 
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Figure 8. PSSA Math Performance: Proficient or Above 


5> Student Achievement 



— — Renaissance Schools Comparison Schools 

All District Schools 

n = 11 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006-10, the percentage of students scoring Proficient 
or above on the Math PSSA was consistently higher among 
the Comparison Schools than at the future Renaissance 
Schools. 

After one year of the Renaissance Schools Initiative 

• On average, the percentage of students at each 
Renaissance School scoring Proficient or above on 
the Math PSSA increased from 30% in 2010 to 
44% in 2011. 


Figure 9. PSSA Math Performance: Below Basic 



Prior to the Renaissance Schools Initiative: 

• From 2006-10, the percentage of students scoring Below 
Basic on the Math PSSA was consistently lower among the 
Comparison Schools than at the future Renaissance Schools. 

After one year of the Renaissance Schools Initiative 

• On average, the percentage of students at each 
Renaissance School scoring Below Basic on 
the Math PSSA decreased from 45% in 2010 
to 32% in 2011. 
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Figure 10. PSSA Reading Performance: Proficient or Above 


Student Achievement 



Renaissance Schools Comparison Schools 

All District Schools 

n = 11 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006 - 2010, the percentage of students scoring 
Proficient or above on the Reading PSSA was consistently 
higher at the Comparison Schools than at the future 
Renaissance Schools. 

After one year of the Renaissance Schools Initiative (2011): 

• On average, the percentage of students at each 
Renaissance School scoring Proficient or above on 
the Math PSSA increased from 24% in 2010 to 
32% in 2011. 


Figure 11. PSSA Reading Performance: Below Basic 


Student Achievement 



Renaissance Schools Comparison Schools 

All District Schools 

n = 11 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• From 2006 - 2010, the percentage of students scoring Below 
Basic on the Reading PSSA was consistently lower at the 
Comparison Schools than at the future Renaissance Schools. 

After one year of the Renaissance Schools Initiative (2011): 

• On average, the percentage of students at each 
Renaissance School scoring Below Basic on 
the Math PSSA decreased from 52% in 2010 to 
41% in 2011. 
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Figure 12. Average Daily Attendance 
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Renaissance Schools Comparison Schools 

n = 11 Renaissance Schools; 72 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


School Attendance 
2006 -2011 



Prior to the Renaissance Schools Initiative 

• In 2006, there was no significant gap between the future 
Renaissance Schools group and the Comparison Schools in 
Average Daily Attendance. 

After one year in the Renaissance Schools Initiative 

• On average, the Average Daily Attendance at 
Renaissance Schools increased from 90% in 2010 to 
92% in 2011. 
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Descriptive Performance of Promise Academy High Schools 

The figures presented on the following pages provide descriptive comparisons of the six-year 
performance of Promise Academy High Schools and a comparison group of high schools along each of 
the seven outcome measures reviewed above for the elementary and middle schools. The Promise 
Academy High Schools were not included in the inferential analysis presented in the preceding analyses 
for the following reasons: 

1) The general non- equivalence of K-8 schools and high schools makes combining the performance 
of 11 th graders with that of 3 rd - 8 th graders misleading since, traditionally, high schools respond 
quite differently to reform efforts than K-8 schools. High schools are more complex institutions, 
and historically have had a much worse track record than K-8 schools in school turnaround 
initiatives. 19 

2) With only two Promise Academy High Schools, there was not a large enough ‘treatment group’ 
to perform a separate high school impact analysis for the Renaissance Schools Initiative. 


Overall, the Promise Academy high schools did not experience the discernible improvements in 
student achievement and attendance seen in the elementary and middle schools in Year One of the 
Renaissance Schools Initiative. 


While the observed performance of the Promise Academy High Schools does not show similar gains to 
those of the elementary and middle schools, an initial year of descriptive findings should not be 
considered a “failure” of the Initiative at the high school level. Rather, these findings simply suggest that 
in Year One, Promise Academy High Schools did not show gains; this is consistent with previous 
research on school turnaround initiatives at the high school level. 


19 de la Torre, M., Allensworth, E., Jagesic, S., Sebastian, J., Salmnowicz, M., Meyers, C., & Gerdeman, R. D. Turning around 
low-performing schools in Chicago. Chicago: University of Chicago Consortium on Chicago School Research. 
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Figure 13 . PSSA Math Scale Scores 


> Student Achievement 

< PSSA Math: Proficient or Above (Grade 1 1 ) 



n = 2 Renaissance High Schools; 18 Comparison High Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• On average, Renaissance High School 11 th grade students’ 
PSSA Math scale scores had been slightly lower than 
students in Comparison High Schools prior to the 
intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• The Comparison High Schools continued to trend 
slightly higher in PSSA Math scale scores. 
Similarly, the Renaissance High Schools show 
slight gains in Year One of the Renaissance Schools 
Initiative. 


Figure 14 . PSSA Reading Scale Scores 


Student Achievement 



Renaissance High Schools Comparison High Schools 

n = 2 Renaissance High Schools; 18 Comparison High Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• On average, Renaissance High School 11 th grade students’ 
PSSA Reading scale scores had been slightly lower than 
students in Comparison High Schools prior to the 
intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• The Comparison High Schools continued to trend 
slightly higher in Reading PSSA scale scores. 
Similarly, the Renaissance High Schools show 
slight gains in Year One of the Renaissance Schools 
Initiative. 
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Figure 15. PSSA Math Performance: Proficient or Above 


> Student Achievement 

< PSSA Math: Proficient or Above (Grade 1 1 ) 



n = 2 Renaissance High Schools; 18 Comparison High Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• On average, Renaissance High Schools had a slightly lower 
percentage of 11 th grade students’ scoring Proficient or above 
on the Math PSSA than the Comparison High Schools prior 
to the intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• Both the Renaissance and Comparison High Schools 
trend slightly higher in the percentage of 11 th grade 
students’ scoring Proficient or above on the math 
PSSA, while the Renaissance High Schools 
continued to have fewer students scoring Proficient 
or above in Year One of the Initiative. 


Figure 16. PSSA Math Performance: Below Basic 


Student Achievement 



— — Renaissasnce High Schools — — Comparison High Schools 
All District High Schools 

n = 2 Renaissance High Schools; 18 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• On average, Renaissance High Schools had a slightly higher 
percentage of 11 th grade students’ scoring Below Basic on the 
Math PSSA than the Comparison High Schools prior to the 
intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• Both the Renaissance High Schools and the 
Comparison High Schools continue to trend slightly 
lower in the percentage of 11 th grade students’ scoring 
Below Basic on the Math PSSA, while the Renaissance 
High Schools continued to have more students scoring 
Below Basic in Year One of the Initiative. 
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Percentage of Students Proficient or Above 


Figure 17 . PSSA Reading Performance: Proficient or Above 


Student Achievement 
PSSA Reading: Proficient or Above (Grade 11) 



Renaissasnce High Schools Comparison High Schools 

All District High Schools 

n = 2 Renaissance High Schools; 18 Comparison Schools 
Renaissance Initiative, Year One: 2010-1 1 


Prior to the Renaissance Schools Initiative: 

• On average, the Renaissance High Schools had a slightly 
lower percentage of 11 th grade students scoring Proficient or 
above on the Reading PSSA than the Comparison High 
Schools prior to the intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• Both the Renaissance and Comparison High Schools 
trend slightly higher in the percentage of 11 th grade 
students’ scoring Proficient or above on the Reading 
PSSA, while the Renaissance High Schools continued 
to have fewer students scoring Proficient or above in 
Year One of the Initiative. 


Figure 18 . PSSA Reading Performance: Below Basic 


Student Achievement 



Renaissasnce High Schools — 

— Comparison High Schools 

All District high ocnoois 



n = 2 Renaissance High Schools; 18 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• On average, the Renaissance High Schools had a slightly 
higher percentage of 11 th grade students scoring Below Basic 
on the Reading PSSA than the Comparison High Schools 
prior to the intervention. 

After one year of the Renaissance Schools Initiative (2011): 

• The Renaissance High Schools and the Comparison 
High Schools continue to trend slightly lower in the 
percentage of 11 th grade students’ scoring Proficient 
or above on the Reading PSSA; the Renaissance High 
Schools continued to have more students scoring 
Below Basic in Year One of the Initiative. 
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Figure 19. Average Daily Attendance 


School Attendance 
2006 - 2011 



Renaissasnce High Schools Comparison High Schools 


n = 2 Renaissance High Schools; 18 Comparison Schools 
Renaissance Initiative, Year One: 2010-11 


Prior to the Renaissance Schools Initiative: 

• Since 2009, Average Daily Attendance the Renaissance High 
Schools has been steadily increasing, while enrollment at the 
Comparison High Schools remained flat. 

After one year of the Renaissance Schools Initiative (2011): 

• In 2011, the Renaissance High Schools closed the gap 
on Average Daily Attendance to become roughly 
equivalent to the Comparison High Schools. 
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Descriptive Comparisons of K-8 Renaissance School Operators 

The following figures present descriptive comparisons of the performance of Renaissance School 
operators - including Charter providers and District-run Promise Academies - to demonstrate the 
relative equivalence of the gains experienced at these schools in Year One of the Initiative . 20 

As mentioned previously in the review of the multi-level modeling results, there were no significant 
differences between the Renaissance Charters and Promise Academies in their performance along any 
outcome measure in Year One of the Initiative. The figures below provide descriptive verification of the 
‘across the board’ gains experienced by all elementary and middle schools in Year One of the 
Renaissance Initiative. Model results testing the significance of differences between Promise Academies 
and Renaissance Charters can be found in Appendix C. 


20 Promise Academy high schools are not included in these figures. See previous section for explanation for their exclusion 
from these comparisons. 
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Average PSSA Math Scaled Score 


Figure 20. PSSA Math Scale Scores 


Renaissance School Operators 

PSSA Math Scaled Scores 



Mastery ■ Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar gains in their PSSA Math scale 
scores. 


Figure 21. PSSA Reading Scale Scores 


Renaissance School Operators 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar gains in their PSSA Reading scale 
scores. 
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Percentage of Students Proficient or Above 


Figure 22. PSSA Math Performance: Proficient or Above 


Renaissance School Operators 


PSSA Math Performance: Proficient or Above 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar gains in the percentage of their 
students scoring Proficient or above on the PSSA Math 
test in Year One of the Initiative. 


Figure 23 . PSSA Math Performance: Below Basic 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar reductions in the percentage of 
their students scoring Below Basic on the PSSA Math 
test in Year One of the Initiative. 
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Percentage of Students Proficient or Above 


Figure 24 . PSSA Reading Performance: Proficient or Above 


Renaissance School Operators 


PSSA Reading Performance: Proficient or Above 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar gains in the percentage of their 
students scoring Proficient or above on the PSSA 
Reading test in Year One of the Initiative. 


Figure 25 . PSSA Reading Performance: Below Basic 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made substantial 
and roughly similar reductions in the percentage of 
their students scoring Below Basic on the PSSA 
Reading test in Year One of the Initiative. 
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Average Daily Attendance Rate 


Figure 26 . Average Daily Attendance 


Renaissance School Operators 

Average Daily Attendance 



Mastery Universal 

Young Scholars Aspira 

Promise Academies 

n = 7 Charters; 4 Promise Academies 
Renaissance Initiative, Year One: 2010-11 


After one year of the Renaissance Schools Initiative: 

• Every Renaissance School operator made roughly similar 
gains in their overall school attendance in Year One 
of the Initiative. 
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Summary of Year One Outcomes 

Overall Performance 

• During Year One of the Initiative, K -8 Renaissance School performance improved significantly 
more than the Comparison Schools across every Student Achievement and Attendance outcome. 

o While group differences between all Renaissance Schools and the Comparison Schools 
were significant, there were no significant differences between the performance of 
Renaissance Charters and Promise Academies. 

• Renaissance High Schools did not experience gains similar to those observed in Renaissance 
Elementary and Middle Schools. 

Academic Achievement 

• In 2010-11, K -8 Renaissance Schools achievement levels increased significantly more than the 
Comparison Schools for every outcome: 

o Math and Reading Scale Scores; 
o Math and Reading Students Proficient or Above; and 
o Math and Reading Students Below Basic 

• Renaissance Schools made greater Math gains in Year One than Reading gains. 

o This pattern is consistent with other school turnaround results in which larger math gains 
are typically observed . 21 


Against Comparison Schools, Renaissance Schools made gains at both ends of the achievement 
distribution on both the Math and Reading PSSAs. 

There were significant increases in the percentage there were significant decreases in the percentage 

of students scoring Proficient or above and... of students scoring Below Basic. 


Attendance 

• On average, Renaissance Schools experienced a significant increase in attendance rates in 2010- 
2011 as compared to the Comparison Schools. 

Enrollment 

• In Year One of the Renaissance Initiative, enrollments at Renaissance Charters increased, while 
enrollments at Promise Academies and the Comparison Schools continued to decline. 

These findings suggest that the Renaissance Schools Initiative is having a strong, positive effect on K -8 
students and schools, in both the Renaissance Charters and the Promise Academies. 


21 de la Torre, M., Allensworth, E., Jagesic, S., Sebastian, J., Salmnowicz, M., Meyers, C., & Gerdeman, R. D. Turning around 
low-performing schools in Chicago. Chicago: University of Chicago Consortium on Chicago School Research; Fryer, R.G. 
(2011). Creating “No Excuses” (Traditional) Public Schools: Preliminary Evidence from an Experiment in Houston. (NBER 
Working Paper 17494). Cambridge, MA: National Bureau of Economic Research. Retrieved from 
http : / / www.nber.org/papers/w17494 . 
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However, although these schools made progress in Year One of the Initiative, they remain among the 
lowest performing schools in the District. Despite the gains reported here, on average, fewer than half of 
students at the Renaissance Schools and the Comparison Schools scored proficient or above on the 
Math PSSA, and just over half of students were Proficient or above in Reading. Indeed, both groups, 
Renaissance Schools and the Comparison Schools, remain substantially below the District levels of 
proficiency in both Math and Reading, and continue to have a higher share of their students scoring 
Below Basic in both Math and Reading. In addition, Renaissance High Schools did not experience 
similar gains to those seen in the elementary and middle schools in Year One of the Initiative. 
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Year Two: A Closer Look at Leadership and Instruction at 
Two Promise Academies 

The summer of 2011 was a tumultuous period for the District, as the timeline on page 3 suggests. In 
May, the School Reform Commission (SRC) passed a budget that called for an unprecedented $629 
million reduction in staffing and supports District-wide. Additional cuts were made in December 2011, 
including two percent from Promise Academy operating budgets and closure of the Promise Academy 
division office. Importantly, this chapter is based on fieldwork completed before this most recent round 
of cuts. 

The summer 2011 budget reductions included seniority-based teacher layoffs across all District- 
managed schools, except for the Promise Academies. This led the Philadelphia Federation of Teachers 
(PFT) to file a lawsuit against the District for violations of their contract. 22 In mid- August, the PFT and 
District settled out of court, with the District agreeing to apply the seniority-based layoff policy equally 
across all schools, including Promise Academies. On August 22, after a summer marked by tension and 
criticism, Superintendent Arlene Ackerman - the initial champion behind the Renaissance Schools 
Initiative - resigned after the city of Philadelphia and the District jointly bought out her contract. 

Meanwhile, in response to the District’s mounting budget crisis, the SRC approved a decision to reduce 
the number of second-cohort Promise Academies from eleven to three, and to make cuts to the Promise 
Academy model. As Figure 27 illustrates, several components of the model that made Promise 
Academies unique, such as extended learning time and a special summer orientation for teachers, were 
reduced. * 2 s 

Despite these changes in the District, Promise Academies still entered Year Two funded at higher levels 
than other Empowerment Schools, and still receiving priority for services and support from District 
offices. For example, the District was still facilitating Promise Academy principal meetings and 
conducting regular walkthroughs. Schools were guided by the Promise Academy Way and were staffed 
by principals and teachers specifically selected for the task of turnaround. The start-up of Year Two was 
challenging, but as one District official observed: 

The constraints that were placed on us because of the context in which we work were 
extraordinary , but given those constraints, I think our leadership team— and that’s the 
principals and school-based leadership teams— did a phenomenal job to make sure they 
did not miss a beat for the youngsters. 


22 The positive relations forged between the District and the PFT when they signed the CBA had soured during Year One when 
the District announced the Renaissance Initiative would include seven charter schools, whose teachers would not belong to the 
union. Furthermore, the PFT was disappointed when they were not invited to the table to partner in designing the Promise 
Academy interventions. 

2 3 The Promise Academy Way, a “leadership handbook” developed by the District in the fall of Year One, outlines expectations 
for Promise Academies in the following areas: vision and mission, climate and culture, systems and procedures, academic 
programs, and parental partnerships. 
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Figure 27 . Promise Academy Models - Year One and Two 
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Year Two Promise Academy Model 



Empowerment 
School Curriculum 

Principal with a 
clear mission & high 
expectations 
for the school 

College going 
Culture 

Increased level 
of support & 
accountability from 
the District 
(e.g. walkthroughs, 
principal meetings) 

Weekly professional 
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common planning 
time 

Reconstitution of 
teaching staff 
(in new Promise 
Academies) with Site 
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Uniforms 

Promise Academy 
Way guiding 
instruction & school 
climate 

School Advisory 
Council 

Reduced funding 
but more than 
other Empowerment 
schools 

Extended day 
reduced from 4 to 3 
days/week 

Fewer Support 
Staff 

NO School-Based Instructional Specialist (SBIS) 

NO Extended Learning time (Saturday School and Summer Academy) 
NO summer orientation for teachers 


As the schools entered Year Two, the media continued to follow the Renaissance Schools Initiative, and 
a variety of community and local reform groups had their eyes on the Initiative and the District’s 
decision to continue funding. Our research over the summer and into the fall suggests that Promise 
Academy staff maintained the notion that they were part of a broad turnaround effort. The sense in 
Year One that the Promise Academies had to “invent as you go” was replaced in Year Two by a calmer 
climate, familiarity with instructional programs, and enthusiasm for growth that students had made in 
Year One. We heard optimism from both teachers and principals. “I think it is just going to keep getting 
better,” said one teacher. “I don’t see it regressing at this point. We have come a really long way.” 
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Case Study Research Methods 

In fall 2011, RFA began the first phase of our proposed 16-month case study research. According to our 
design, we would look more deeply at the turnaround process in two Promise Academies and two 
charter-managed schools, selected from the first 13 Renaissance Schools. With the support of the 
District, we selected two Promise Academies; to date, none of the Renaissance charter providers have 
agreed to allow their schools to participate. The purpose of our case study research is to identify 
emergent practices that are contributing to accelerated school improvement, as well as barriers 
impeding the schools’ success. The following questions guide the case study research: 

1) How do schools vary along the three key drivers of turnaround— Effective School Leadership 
and Staffing, Positive School Climate, and Parent and Community Engagement? 

a. What best practices are emerging that contribute to strengthening schools along these 
dimensions? 

2) What district and contextual conditions contribute to the Renaissance reform model working 
most effectively? What district and contextual conditions prevent Renaissance Schools from 
functioning more effectively? 

The two Promise Academy schools selected for study include elementary and/or middle grades, and 
reflect variation in terms of geographic location, student demographics, and early indicators of success 
in areas of academic performance, school climate, and SAC functionality. 2 4 Table 3, below, shows the 
differences between the two schools in relation to the Renaissance Schools average, as of the end of 
Year One. 


T able 3 . Comparison of Case Study Schools to Renaissance Average 


Renaissance School Averages 

Mean 

fstandard deviation) 

School A* 

School B* 

Total School Enrollment 

546 ( 181 ) 

- 0.84 

0.90 

Percent Minority Students 

99 % ( 2 %) 

- 0.67 

0.00 

Percent ELL Students 

6 % ( 9 %) 

- 0.62 

2.13 

Percent Special Ed. Students 

17 % ( 5 %) 

- 0.46 

1.80 

Percent Free Lunch Students 

94 % ( 4 %) 

- 0.75 

0.00 

Percent Proficient: Math PSSA 

39 % ( 16 %) 

0.52 

- 0.13 

Percent Proficient: Reading PSSA 

29 % ( 9 %) 

- 0.11 

0.00 


* Values represent standard deviation units away from the overall Renaissance School group average. 


2 4 We did not select either of the Renaissance high schools for case study because we wanted to be able to make meaningful 
comparisons among case study schools. High schools are complex institutions and respond differently to reform efforts. With 
only two high schools in the first cohort, both Promise Academies, comparisons across case study schools would have meant 
comparing elementary schools to high schools, and our ability to identify promising practices across schools would have been 
limited. We therefore eliminated the two high schools from our case study selection process. For more detail on our process for 
selecting case study schools, see Appendix D. 
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The findings presented in this section of the report are based on the following data sources: 


Table 4. Data Sources 


Timeframe: September 2011 - January 2012 

Interviews 

Document review 

School site visits 

• Six District officials 

• Two PFT 
representatives 

District and school materials 

• Promise Academy Way 

• Grade group meeting agendas 

• School Quality Reviews from 
early 2010 

At each case study school: 

• Interview with principal 

• Interview with teacher leader 

• Two teacher focus groups (new and 
returning) 

• Three classroom observations 

• One grade group meeting 
observation and/or a professional 
development session 


The focus of our fall 2011 fieldwork was on school leadership and instructional capacity 
in two Promise Academies. By the end of Year One, we had noted considerable positive change in 
school climate, and principals had conveyed that they expected Year Two to be one of dramatic 
academic improvement. For this reason, we believed it important to begin our case study work with a 
focus on how school leaders and staff were building capacity to bring about dramatic academic 
improvement.^ In the following sections, we describe promising practices related to leadership and 
instruction at our two case study schools as they entered Year Two, followed by a brief description of 
obstacles the two schools faced in these efforts. 

Promising Practices 

In Year Two, both case study schools remained focused on the goal of turning around low student 
performance. Students and teachers had become familiar with the behavioral standards, academic 
expectations, and prescribed curriculum of the Promise Academy Way. While their structures differed, 
leadership at both schools shared a common focus on improving instruction. Staff at both schools 
believed that building the instructional capacity of their teachers, while holding them accountable for 
rigor and meeting the needs of their students, was the route to student success. 


School principals were the driving force behind the promising practices below. These seven promising 
practices emerged in our research at both case study schools. While we are unable to draw direct links 
to student improvement, our data suggest that these practices are supporting teaching and learning at 
both schools. 


2 s We intend to return in spring 2012 to revisit issues related not only to leadership, staffing and instruction, but also to school 
climate and to continue to follow the development of the SACs. In summer/fall 2012 we would conclude with a last round of 
school interviews. 
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PROMISING PRACTICE: Principals are goal-oriented and cultivate buy-in from staff 


Principals at both schools clearly communicated a mission of high expectations for students and 
teachers. Teachers in both schools echoed the principals’ goals, and respected their principals as strong 
leaders. 


School A 

School B 

Principal: “What you want to look for is student 
growth. Moving students is a success.” 

Principal: “We are offering the accelerated 
courses. ..so that our kids won’t be forced to go to 
the neighborhood high school.” 

Teacher: Our current principal is “a good leader just 
because [his/her] accountability standards are much 
higher than any other principal that I’ve ever had. 

Teacher: “Having the same leader and having 
people in the school who buy into [the principal s] 
vision is what will make this school successful from 
my perspective.” 


PROMISING PRACTICE: Principals site-select. 

At both schools, site selection contributed to a staff who wanted to be part of a change initiative. In 
contrast, teachers who were centrally placed (i.e., not site selected) due to the rushed hiring timeline did 
not all wish to teach at a Promise Academy. 


School A 

School B 

Teacher: “The Promise Academy initiative was 
attractive to me because 1 wanted to help be part of 
an initiative to help turn around schools.. .1 went 
through the selection process for that. It was 
exciting.” 

Principal: Selecting staff is “almost the most 
important element” of the Promise Academy 
model. 


PROMISING PRACTICE: School leaders support teacher growth with ongoing observation, 
assessment, and professional development to improve instruction. 

Both principals believed that teacher development was the pathway to student growth. They directed a 
coordinated system of teacher supports that included weekly professional development, common 
planning time, and regular informal observations, as mandated by the Promise Academy Way. As 
illustrated in Figure 28 below: 

• Principals and their leadership teams used student data, feedback from District walkthroughs, 
and/ or their own observations of classrooms to determine the focus of Monday afternoon 
professional development (PD) sessions prescribed by the Promise Academy Way. 

• Teachers then met weekly as grade groups— also mandated by the Promise Academy Way— to 
discuss the implementation of the strategies or initiatives covered in PD. 
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• In informal classroom observations, principals, assistant principals, and teacher leaders looked 
for implementation of these same strategies, provided teachers with individual feedback, and 
sometimes modeled the new strategy or initiative for the teachers in the classroom. 

This coordinated system provided teachers with continuous feedback on their instruction. In addition 
to their informal observations, school leaders were to conduct at least two formal evaluations annually. 
Teachers at both schools said the regular observations and feedback “kept them on their toes” and 
provided concrete suggestions for improving their practice. 


School A 

School B 

Teacher: “Our principal does informal observations 
and takes notes. You can always go to him and ask 
for his feedback and see if he has ideas about 
implementing lessons.” 

Principal: “If we can constantly provide the support, 
and monitor the growth, and monitor the level of 
support to teachers, [then] in turn student success 
will be as high if not higher than it was on last year’s 
test. ..Everything ties in to the level of teacher 
growth and the level of support that is provided to 
the teaching staff so that they are successful.” 

Teacher: “There is a pretty comprehensive 
feedback form that is used. It is standardized across 
the entire school. It is usually like two goods’ and an 
improve’ is what is listed on there.” 


Figure 28. Mutually-Reinforcing Mechanisms for Improving Instruction 


Principal 


i 

Observe: 

Look for evidence 
of strategies being 
implemented in 
the classroom 

Assess: 

What is needed 
to improve 
instruction? 

Teach: 

Determine focus 
of Weekly 
Professional 
Development (PD) 

Reinforce: 

In Grade Groups 
discuss how to 
implement 
strategies covered 
in PD 


PROMISING PRACTICE: School leaders and teachers use multiple data sources to track student 
progress and adjust instruction. 

Both schools accessed and used multiple forms of data to chart progress and inform instructional 
strategies. Among the types of data mentioned were: 
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• Data from SchoolNet 

• Predictive data 

• PSSA data 

• Formal and informal classroom observations 

• District walkthrough observations 

• Portfolios of student work 

• Anecdotal notes and report card comments made by teachers 

As illustrated by Figure 28, school leaders used these data sources to assess where teachers needed 
support. In addition, teachers used data individually, in grade groups, and in PD sessions to identity 
areas where their students needed reinforcement and to decide how to differentiate instruction. PD 
included sessions focused on how to access data and how to use it to assess students and meet their 
instructional needs. 

PROMISING PRACTICE: Principals encourage teachers to adapt the curriculum within the Promise 
Academy Way. 

Principals and teachers saw the Promise Academy Way and prescribed curriculum as the framework in 
which they were working. In Year One, many teachers felt overwhelmed by the need to learn many new 
curricula and by the constraints of the curricular programs they were to implement. In Year Two, 
however, they were beginning to feel familiar enough with the curriculum to start adjusting their 
lessons where they saw limitations. 

Both principals were introducing strategies that lessened direct instruction. Adjustments included: 

• Emphasis on writing and the use of writing portfolios; 

• More time spent in guided reading; 

• Use of choice boards; and 

• Teacher-designed activities to support students at different levels so, in the words of one 
teacher, “all levels of students can understand the material.” 


School A School B 


Principal: “I don’t want too much direct instruction. 
Some teachers are good at it because they’re 
forceful and they have a presence. Some teachers, 
the kids don’t even know they are in the room, so 
direct instruction doesn’t work for them. I’d rather 
they have kids in groups working independently on a 
project.” 


Teacher Leader: “So whatever your creativity is, 
you’re working within that structure [of the Promise 
Academy Way]. ..How much student voice am I 
allowing? Am I making sure that I am hitting the 
eligible content? Am I doing the same thing every 
week or am I keeping it interesting for the students? 
So there is ownership of that process and that falls 
within the Promise Academy Way, because 
ultimately we want to move all of the kids.” 
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PROMISING PRACTICE: Principals encourage teacher collaboration. 

Both principals encouraged teachers to share ideas with each other - both informally and during the 
formal grade group meetings. Teachers met before and after school, during common prep periods, and 
during lunch. Some also said they texted, called, and emailed each other at night and on the weekends 
to sustain the collaboration. One principal encouraged teachers to use a prep period to observe a 
teacher who was strong in an area where they struggled. 


School A 

School B 

Teacher: “A lot of times, we ll talk before school. 
We ll talk after school. We talk at lunch time. ..We 
text and email. It’s not always during school time 
that we communicate. 

Teacher: “People are more willing to share their 
knowledge [now that we are a Promise Academy]. 1 
know there was a time here when teachers couldn’t 
even knock on each other’s doors because [they 
thought] j have this and I’m keeping it,’ but now it is 
about sharing. ..It is to increase the knowledge base 
of the students, and most of the staff is really willing 
to do that, and to follow that directive from the 
principal.” 


PROMISING PRACTICE: The PAR program supports ongoing PD. 

The Peer Assistance and Review program (PAR) matches struggling teachers with teacher mentors, and 
has strong, experienced teachers design and carry out research projects. At School B, teachers who were 
matched with PAR mentors spoke highly of them. They valued the “outside, unbiased opinion” of their 
mentors. School A had experienced teachers working on research projects, but the mentoring program 
was not yet in place. The principal planned to implement it next year for teachers getting unsatisfactory 
write-ups. 


School A 

School B 

Principal: “1 do like that [the PAR program] is not 
punitive. It’s meant to support the teacher and help 
the teacher.” 

Teacher: “1 got the PAR support program. That has 
been phenomenal. ... [My mentor teacher] has been 
great. If 1 didn’t have her 1 think 1 would have been 
completely clueless. ... That’s where 1 have gotten 
most of my classroom changing advice from. 


Teacher Satisfaction 

With these promising practices in place, and one year behind them, teachers on the whole spoke with 
more optimism and focus than they had in the fall of Year One: 

Teachers felt like they were part of something “big.” Given (l) the attention the Renaissance 
Schools Initiative received from the District— including its status as the signature reform of former 
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superintendant Arlene Ackerman; (2) the way the Initiative has been spotlighted in the local media; and 
(3) the federal government’s promotion of school turnaround efforts, teachers were energized by the 
sense that they were participating in something beyond the scope of their individual classrooms. 

Teacher morale in our two case study schools was higher among teachers who were 
returning for their second year at the same school. When we conducted interviews in the fall of 
Year One, teachers in our case study schools spoke of being overwhelmed and frustrated by constant 
changes. In Year Two, returning teachers appeared to be familiar with the Promise Academy Way, 
expressed confidence in school leadership, and felt encouraged by evidence that their schools had made 
improvements in Year One. 


School A 

Teacher: “What made me stay was, I actually did 
see a difference, at least from the beginning of the 
year to the end of the year. Just talking to other 
teachers, [they said ] my, have things changed since 
last year.’ So that was a good sign. I know that things 
take time to evolve. So I thought next year would be 
even better; we would see even more progress.” 


Teacher morale was not as high among those who were new to the school in Year Two. 

Principals worked hard to help acclimate these teachers, many of whom arrived at a Promise Academy 
because they had been force-transferred from elsewhere in the District. 


School B 

Teacher: “I think what I am seeing with the teachers 
that were, I guess, transferred from other schools ... 
was some of them worked at other schools six years, 
five years, or two years, even -and they come here 
and they don’t want to work at a Promise Academy. 
They let it be known, they tell you that, and you see 
it in their actions and you can see it in their voices 


Obstacles to Improvement 

While principals and teachers were instituting these promising practices and expressing optimism that 
they would continue to see improvement in their schools in Year Two and beyond, they also identified 
factors that may hinder progress. A number of the obstacles they described were the result of turmoil 
and change at the District level. 
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CHALLENGE: Staff reductions. 

At School A, staff reductions meant the end of small group pull-outs, an increase in classroom size 
(from under 25 to 25-30 students), and elimination of in-school substitutes. At School B, special 
education teachers were cut (from 15 to 9), which limited services for those students. 


School A 

School B 

Teacher: “You can’t have a turnaround school and 
pile 25 needy, needy [kindergartners] into one room 
and expect miracles to happen.” 

Principal: “Last year we were able to have at least 
two or three learning support teachers in every 
grade which allowed time for the push-in and pull- 
out in small group sessions. This year we were not 
afforded that opportunity so in a class room you are 
seeing 15 kids per one special education teacher.” 


CHALLENGE: Teacher turnover. 

Because of their high number of new teachers in Year One, both schools reported teacher turnover of 
25-40% following District-PFT settlement and subsequent seniority-based layoffs that occurred in mid- 
August 2011. The turnover delayed principals’ efforts to achieve common vision and norms among their 
staff. Teachers new to the school in Year Two were not acclimated to the principals’ turnaround mission 
and high expectations. 


School A 

School B 

Principal: “You know they wiped us out, with the 
teachers. 1 had to begin to get teachers back. At the 
very last minute, 1 was able to get a lot of my 
teachers back who were here last year but then on 
the other hand, 1 didn’t get all my teachers back.” 

Principal: “Now the teachers that are new to us, 
most are at the beginner stage because what good 
instruction looks like at [our school] might not have 
looked the same in their former school. Things that 
are monitored here may have never been monitored 
at their former school. So as you’re training and 
building them up through the scaffolding approach, 
you’re also breaking bad habits.” 


CHALLENGE: Late hiring and no orientation. 

Principals had less than a month to fill vacancies following the August 2011 layoffs. As a result, the 
hiring process was rushed and site selection was compromised. Most of the available teachers were 
forced transfers that did not necessarily want to be at Promise Academies. One principal interviewed 
candidates but did not have time to conduct observations. The other school still had over fifteen 
vacancies after site selection had ended, which were filled centrally. Site selection was a promising 
practice in both schools (see above), and principals said that having it undercut in this way slowed 
improvement at their schools. 
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The combination of late hiring and budget cuts meant there was no summer orientation for the new 
teachers. In one school, teacher assignment did not settle until October, which led one teacher to 
describe the first month of school as “a ball of confusion.” 


CHALLENGE: Over use of corrective reading and math. 

Teachers at both schools said that Corrective Reading and Corrective Math should be used more 
selectively with students showing need. They did not see positive gain from its wholesale use. The 
critique of Corrective Math was especially common . 26 


School A 

School B 

Teacher Leader: “1 think it should be used like a 
Band-Aid, and not across the board. ... 1 think the 
way it is implemented, it’s not really that effective.” 

Teacher Leader: “Honestly kids don’t love it, 
teachers don’t love it, and 1 think that it is great for a 
specific population, as that’s how it was designed ... 
but it’s being utilized as an intervention for 
everybody.” 


CHALLENGE: Over- reliance on direct instruction. 

The prescribed curriculum for Promise Academies calls predominantly for direct instructional 
approaches. In response, principals at both schools were introducing alternative strategies that 
encouraged student-centered instruction and small group work. Their efforts to adapt the curriculum 
were described as a promising practice. 

Discussion of Case Study Findings 

Our close look at two Promise Academies in Year Two of the Renaissance Schools Initiative highlights 
the important role played by school principals. School leaders articulated a clear mission and goals for 
the school, and they understood that teacher development had to be the primary pathway to student 
growth. Even in the face of challenges created by budget cuts, teacher layoffs, and restrictive curricular 
demands, principals believed they could build momentum through formal and informal systems that 
supported teachers’ learning. In the words of one District leader, principals were “resilient” in the face 
of curtailed resources, and they and their staff were working to improve outcomes for students. 

As of this writing, however, additional reductions are being made to the Promise Academy model. 
District and union officials reported that the reductions and general uncertainty in the Promise 
Academies are jeopardizing morale in the schools. One District leader questioned how much it still 
stands apart from the Empowerment School model, and whether it will result in the growth they hope 
for: 


26 On February 13, 2012, the District announced to the SRC that beginning as soon as fall 2012, they will no longer mandate 
use of Corrective Reading and Corrective Math. See: Herold, B. (2012, February 13). District to stop mandating use of scripted 
curricula. The Philadelphia Public School Notebook. Retrieved from http://www.thenotebook.org. 
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The model needs more clarity; we need to know what it is and the schools need to know what is 
extra for them. They still have some additional time, they do have priority in some way in the 
District, and they have teachers that have chosen in some way to be there, as opposed to being 
assigned there, though that did happen a bit at the end. I am a little frustrated and 
disappointed with where the model is. I don’t think it’s where we intended it to be. I think we 
will see growth and improvement, but if we wanted to see accelerated growth— I’m frustrated. 

While the future of the Promise Academies remains uncertain, the promising practices identified here 
are supported by existing research on leadership practices and instructional systems that lead to 
teacher and student growth. 2 ? These practices, when well-implemented, hold the potential to improve 
teaching and learning not only in the Renaissance Schools but in other schools and in other districts. 


Conclusion 

As of this writing, the District remains in an unprecedented state of flux. Four of the five SRC members 
are new to their appointments; the SRC is facing the need to make an extraordinary number of budget 
cuts due to a grave shortage of funds; the mayor has taken on a new, activist role given the dire state of 
District finances; and a national search is underway for a new superintendent. The process for a third 
round of Renaissance schools started late, in January 2012, but the process is in motion and the plan 
includes converting another set of persistently low-performing District schools to Renaissance Charter 
schools, with no announced plans for new Promise Academies. 

There is little doubt that the Renaissance Schools Initiative was launched under less than ideal 
circumstances, and they continue to operate within a context that is fraught with change and 
uncertainty. Yet despite these circumstances, this interim report shows encouraging signs for the 
Renaissance Schools Initiative in both the Promise Academies and the Renaissance Charters: 


Both models - Renaissance Charters and District-run Promise Academies - made strong positive 
gains toward improving student achievement and school attendance. Gains in the 11 K-8 Renaissance 
schools were significantly greater than those in comparison schools in: 

• Math and Reading PSSA scale scores 

• Percentage of Students Proficient or above in Math and Reading 

• Percentage of Students Below Basic in Math and Reading 

• Average Daily Attendance 

No differences in effectiveness among the five providers (School District and four charter providers) 
were discerned. 


2 7 Christman, J. B., Brown, D., Burgess, S., Kay, J., Maluk, H. P., & Mitchell, C. (2009). Effective organizational practices for 
middle and high school grades: A qualitative study of what's helping Philadelphia students succeed in grades 6-12. 
Philadelphia: Research for Action; Bryk, A., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2010). Organizing 
schools for improvement: Lessons from Chicago. Chicago: University of Chicago Press; Herman, R., Dawson, P., Dee, T., 
Greene, J., Maynard, R., Redding, S., & Darwin, M. (2008). Turning around chronically low-performing schools: A practice 
guide (NCEE #2008-4020). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of 
Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications/practiceguides. 
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This report also highlights promising District policies and school practices that could have importance 
for future turnaround efforts in Philadelphia or in other locales. These include the following: 


1) The Promise Academies received extra resources and attention from the District, which placed 
them in the spotlight and generated grater public interest in their progress. 

2) The teachers’ union supported the Renaissance Schools Initiative by signing a collective 
bargaining agreement that was consistent with the principles of the reform effort. 

3) Principals and teachers felt part of something “big.” 

4) Principals built their own teams of teachers through site selection. 

5) Principals built systems that promoted and reinforced teacher learning and growth. 

6) Data and student work were used to assess learning and make instructional decisions. 

7) Principals and teachers exercised professional judgment to adapt the curriculum, within the 
parameters of the Promise Academy Way. 


Implications and Next Steps 

The Year One outcomes for schools in the Renaissance Schools Initiative suggest that something 
positive is happening in the first cohort of Promise Academies and Renaissance Charters. Year One of 
the Initiative saw significant positive gains in the rate of student achievement and attendance; and Year 
Two case studies point to an emerging set of promising practices that are likely to be related to the 
success of these schools. 

Given these notable changes in Year One, we believe that ongoing research should address the following 
questions: 

1) Does the rate of growth in student achievement at Renaissance Schools continue in subsequent 
years? 

2) Is the model scalable— that is, as it expands to include more schools, can the successes of the 
Year One cohort be replicated? What level of resources or supports is necessary to achieve this 
goal? 

3) What is the cost of implementing successful Renaissance schools using the Promise Academy 
model? The Renaissance charter model? In K-8 schools? In high schools? 

4) Do differences between providers emerge over time? 

5) Does the success of the model differ by type of school (K-8, high school) or student population 
(special education, ELL, low-income, high-performing, low-performing) 

6) At both Promise Academies and Renaissance Charters, what replicable policies and practices are 
contributing to increased student achievement? 

7) What is the collateral effect for comparable District schools not selected for the Renaissance 
Schools Initiative? 
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RFA’s ongoing research will continue to monitor key student outcomes through Year Two of the 
Renaissance Schools Initiative, and our case studies of two Promise Academies will continue through 
the remainder of the 2011-12 school year. The study would be bolstered by the addition of charter 
schools to the case study design to ensure that the promising practices of these schools are also 
documented in our research. We will continue to reach out to charter school operators in the hopes of 
gaining entre to one or two of the Renaissance Charter schools. 
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Appendix A: School Advisory Councils 

This appendix provides a brief update on the development of School Advisory Councils (SACs) in Year 
Two, including a closer look at the SACs in the two Promise Academies we have begun to follow as case 
studies. A more extensive examination of SACs will be part of our December 2012 report. This update is 
based on the following data sources gathered between September 2011 and January 2012: 

• Interviews with 6 District officials 

• Observation of a District-organized meeting for principals and SAC chairs/members 

• Observation of a SAC Working Group meeting 

• Document review of District materials (e.g. SAC Handbook, FAQ Sheet, and Overview of SACs 
handout) 

• Document review of materials produced by SACs and community groups 

• Research activities at each case study school: 

o Interview with principal 
o SAC meeting observation 
o Survey completed by SAC chairs 
o SAC reports on goals to the District 

In the following section, we examine: 

1) An increased level of support for Promise Academy SACs in Year Two 

2) Promising practices in the development of SACs at our two case study schools 

3) Challenges that SACs continue to face. 

We do not report on Charter Renaissance SACs because we did not have access yet to Renaissance 
charter schools. 

Increased Support for SACs in Year Two 

In Year One, we found that multiple school and Central Office staff were providing guidance to the 
Promise Academy SACs, and that the SACs would benefit from a single source of Central Office 
support. 28 Year Two began with guidance to the Promise Academy SACs consolidated under the Office 
of Parent, Family, Community Engagement and Faith-Based Partnerships (OPFCEF). As one member 
of that office commented about its new relationship to SACs: “I’m holding myself personally 
accountable for goals. I want to hear from SACs and principals together. What are the goals? What are 
parents working on? It’s not just attending meetings. How do we measure, what are benchmarks, what 
is the timeline for these goals?” 

To solidify this new relationship with the Promise Academy SACs, the OPFCEF took the following steps: 

• Designated one OPFCEF staff person to provide direct support to each SAC, including helping 
with parent outreach for meetings and attending most SAC meetings; 

• Revised the SAC Handbook; 


28 See: Gold, E., Good, D., Roberston-Kraft, C., & Callahan, M. K. (2011). 
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• Planned an October 2011 SAC Summit “for all members of the SACs as well as for any parents of 
Philadelphia public schools interested in forming a SAC at their school” 29 : 

• Incorporated training on SACs into the agenda for the principals’ retreat; 

• Organized a joint meeting of principals and SAC chairs; 

• Developed a schedule of regular PD sessions for SAC chairs and members; some of these were 
planned for the school site and were to be tailored to the needs of that SAC; and 

• Developed a report template (requesting school data, a SAC self-assessment, and goal setting) 
and required that SACs submit the reports bi-annually; these will presumably be shared with 
District leadership and the SRC. 

In addition, by Year Two, considerable public advocacy had grown in support of SACs— not only in the 
Renaissance schools but throughout the District. The OPFCEF coordinated a SAC Working Group that 
includes members of five to seven advocacy and community and youth organizing groups. The SAC 
Working Group, which has met on a near-monthly basis since the summer, assisted with planning 
trainings, the SAC Summit, and the revised SAC Handbook— including promoting the new requirement 
that high school SACs include three high school students. 

Several of the groups represented in the SAC Working Group are also members of Our City - Our 
Schools, a coalition that is actively advocating for the development of SACs beyond the Renaissance 
schools, and for the strengthening of the decision-making and accountability roles of SACs. According 
to their tally, 55 District schools have SACs thus far and at least 18 more are working on creating one, 
suggesting that this component of the Renaissance Schools Initiative is already being replicated in other 
settings. The coalition has developed a platform, which calls on the District to “keep and strengthen 
SACs,” and is meeting with District leaders and SRC members, as well as with city officials, to build a 
shared understanding of what it would take to create strong parent, youth, and public engagement 
through the SACs. 

Promising Practices in Year Two SACs 

Table At points to several promising practices that emerged in our case study research at two Promise 
Academies. Overall there were differences between the two SACs, but both were more organized and 
purposeful than in Year One, and had improved relationships with their principals. They were primarily 
focused on building relationships with parents and recruiting them to participate in their efforts. One of 
the two also articulated a clear mission of developing their school as a hub of services for parents, 
families, and the neighborhood at large. 


2 9 Quoted from the District website. 
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Table Ai: Promising Elements in the Development of the SACs 


Promising Practice 

In the Case Study Schools 

Experienced SAC 
leadership 

At least one SAC chair or co-chair at each case study school 
remained in place in Year Two and was a school parent. Continuity 
in leadership appeared to build on the experience of Year One, 
including an improvement in relationships with their respective 
principals. 

Active SACs 

Although it was too early to tell if the SACs were fully functional and 
what their strongest contributions to their schools were, SACs at 
both schools were meeting regularly and organizing a variety of 
activities for parents and community members. 

Increased principal 
engagement 

In Year Two, both case study SACs appeared to be working 
cooperatively with their principals. One SAC chair said the 
relationship had significantly improved from last year. SAC chairs at 
both schools reported that their principals attended SAC meetings 
more than half the time. A District official commented that now that 
principals realize SACs “are not going away,” communication in 
general is improving. A principal new to his school— a non-case 
study school— noted at a city-wide meeting: “Working with the SAC 
is how I was able to get the community to accept the transition. The 
SAC has been instrumental in allowing me to build that community 
relationship that is so important to the school.” 

Strong focus on outreach 

A primary activity of the SACs at both schools was parent and 
community outreach. One principal noted how important it was for 
SACs to be proactive in recruitment: “They initiate a lot of things. 
They have computer and health workshops. All of those things bring 
parents in, and then you can do your recruiting.” This SAC was 
looking as well for opportunities to build partnerships with 
community organizations and local churches to increase resources 
available to the school and neighborhood families. They were 
focused, in their words, on “Build[ing] the community by addressing 
issues that create barriers within the lives of the families of our 
students to in turn create environments that foster learning and 
success.” 


Continuing Challenges 

The challenges that emerged in our Year Two research echo those we reported on in Year One with a 
few changes. For example, while we have consistently found that building positive relationships 
between SACs and principals has been challenging, we heard a new need in Year Two for collaboration 
among SACs and SAC chairs. As we have reported in the past, recruiting parents and obtaining 
resources for planning their activities are ongoing needs for SACs. In our observations, these needs 
sometimes became the SACs’ primary agenda items, overriding their role in providing, as the SAC 
Handbook describes it, “a real voice in school-based decision-making.” 
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Table A2: Obstacles to Improvement 


Challenge 

The challenge in schools 

Some new SAC chairs 
conveyed a sense of feeling 
isolated. 

We observed in a District-wide meeting that new SAC chairs 
wanted opportunities to learn and share with other SAC chairs, 
especially around issues such as needing to recruit parents. 

Concept of shared leadership 
is not a familiar one. 

Year Two began with better principal-SAC relations in both 
case study schools, but the concept of shared leadership, and 
the role of the SAC in decision-making, remains an area for 
exploration. One District staff reflected: “How do school 
administrators and staff collaborate with parents and 
community around school improvement goals? How do they 
really share leadership and share decision-making so that 
voices are represented at the table and all of the stakeholders 
have a say and that it gets the consensus?” 

Sustaining parent engagement 

Parent participation in SACs continued to be sporadic rather 
than sustained in many of the Promise Academy SACs, and 
SACs were continuously needing to initiate activities that 
would attract parents and community members. Both case 
study SACs were planning events for parents in hopes of 
recruiting new members. 

Resources for refreshments, 
childcare, and translation 
services and printing 

Obtaining a small amount of funding for basic expenses 
remained a challenge for most SACs. Both case study SACs 
reported that they had no money to spend on their activities. 


In Year Two, our case study SACs showed evidence that they were now established, had worked through 
some of the early tensions with their principals, and were clearly focused on reaching out to school 
community members and parents. The progress we saw in the two case study SACs may portend a role 
for SACs that not only aims to influence decisions made inside their school but strengthens the 
neighborhood as a whole. One SAC in particular was addressing the intersection of interests between 
school and community— holding workshops on truancy and attendance, and facilitating services to 
community members in areas related to health, housing and education. 

The fact that SACs are being implemented in schools beyond the Renaissance Schools Initiative holds 
promise for ongoing strides forward in school-community relations. Future research, however, is 
needed to examine the range of ways in which SACs are contributing to neighborhood and school 
improvement, and to develop qualitative and quantitative measures of their contributions. 
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Appendix 6: Supplemental Tables 

Renaissance Schools: School by School Performance 

Table Bi 


Dr. Ethel Allen School K-6 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1127.8 

1135.5 

1156.9 

1168.0 

1170.5 

1240.8 

Percent Proficient or Above 
in Math 

29 . 0 % 

21 . 4 % 

25 . 6 % 

24 . 2 % 

32 . 2 % 

46 . 6 % 

Percent Below Basic in Math 

46 . 2 % 

49 . 5 % 

49 . 4 % 

44 . 2 % 

41 . 7 % 

29 . 3 % 

PSSA Reading Average 
Scale Score 

1053.3 

1128.2 

1128.8 

1111.1 

1115.3 

1175.4 

Percent Proficient or Above 
in Reading 

14 . 2 % 

17 . 1 % 

21 . 6 % 

179 % 

24 . 1 % 

279 % 

Percent Below Basic in Reading 

62 . 3 % 

59 . 5 % 

50 . 6 % 

56 . 8 % 

51 . 8 % 

38 . 0 % 

Enrollment 

389 

394 

336 

342 

390 

394 

Student Daily Attendance 

89 . 1 % 

89 . 3 % 

89 . 2 % 

89 . 1 % 

91 . 4 % 

91 . 6 % 


Roberto Clemente Middle School 5-8 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1162.7 

1160.5 

1208.0 

1231.2 

1201.9 

1237.5 

Percent Proficient or Above 
in Math 

18 . 7 % 

21 . 1 % 

29 . 2 % 

33 . 8 % 

27 . 2 % 

37 . 3 % 

Percent Below Basic in Math 

55 . 3 % 

56 . 9 % 

47 . 6 % 

41 . 1 % 

49 . 6 % 

40 . 3 % 

PSSA Reading Average 
Scale Score 

1076.3 

1049.3 

1115.0 

1114.5 

1123.0 

11771 

Percent Proficient or Above 
in Reading 

18 . 1 % 

17 . 7 % 

22 . 5 % 

23 . 5 % 

25 . 1 % 

29 . 4 % 

Percent Below Basic in Reading 

60 . 8 % 

60 . 6 % 

54 . 3 % 

52 . 1 % 

52 . 1 % 

41 . 7 % 

Enrollment 

1265 

1073 

865 

694 

693 

708 

Student Daily Attendance 

86 . 9 % 

87 . 5 % 

88 . 8 % 

88 . 0 % 

88 . 6 % 

90 % 
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Paul L. Dunbar School K-8 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1170.1 

1109.3 

1103.6 

1135.8 

1159.1 

1236.7 

Percent Proficient or Above 
in Math 

24 . 3 % 

15 . 9 % 

15 . 3 % 

18 . 0 % 

24 . 5 % 

45 . 8 % 

Percent Below Basic in Math 

44 . 4 % 

65 . 0 % 

65 . 3 % 

53 . 9 % 

49 . 1 % 

23 . 4 % 

PSSA Reading Average 
Scale Score 

1123.6 

1116.0 

1088.4 

1106.6 

1135.8 

1230.2 

Percent Proficient or Above 
in Reading 

22 . 3 % 

18 . 1 % 

16 . 6 % 

17 . 1 % 

19 . 8 % 

38 . 3 % 

Percent Below Basic in Reading 

53 . 0 % 

60 . 6 % 

67 . 5 % 

65 . 9 % 

57 . 5 % 

25 . 2 % 

Enrollment 

291 

266 

235 

214 

180 

169 

Student Daily Attendance 

93 . 4 % 

91 . 1 % 

88 . 8 % 

89 . 7 % 

90 . 8 % 

92 . 0 % 


T homas Potter School K-8 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1121.0 

1112.6 

1165.5 

1186.5 

1201.3 

1245.6 

Percent Proficient or Above 
in Math 

20 . 7 % 

17 . 2 % 

26 . 2 % 

30 . 5 % 

31 . 9 % 

39 . 4 % 

Percent Below Basic in Math 

58 . 9 % 

62 . 5 % 

56 . 3 % 

45 . 8 % 

43 . 5 % 

34 . 7 % 

PSSA Reading Average 
Scale Score 

1053.5 

1075.7 

1112.9 

1126.0 

1157.2 

1175.5 

Percent Proficient or Above 
in Reading 

18 . 0 % 

15 . 0 % 

22 . 2 % 

24 . 5 % 

29 . 1 % 

33 . 7 % 

Percent Below Basic in Reading 

63 . 5 % 

66 . 8 % 

59 . 9 % 

51 . 7 % 

48 . 2 % 

44 . 7 % 

Enrollment 

607 

568 

516 

471 

474 

414 

Student Daily Attendance 

88 . 3 % 

89 . 5 % 

89 . 6 % 

90 . 6 % 

90 . 0 % 

90 . 3 % 



University City High School 9-12 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1043.9 

1042.7 

1017.7 

1056.1 

1011.9 

1050.3 

Percent Proficient or Above 
in Math 

9 . 2 % 

3 . 2 % 

5 . 9 % 

8 . 4 % 

3 . 1 % 

9 . 0 % 

Percent Below Basic in Math 

78 . 6 % 

76 . 1 % 

82 . 0 % 

79 . 9 % 

85 . 1 % 

79 . 5 % 

PSSA Reading Average 
Scale Score 

1061.0 

1018.1 

1033.3 

1035.4 

970.3 

1027.2 

Percent Proficient or Above 
in Reading 

16 . 1 % 

10 . 4 % 

11 . 6 % 

11 . 7 % 

6 . 3 % 

13 . 5 % 

Percent Below Basic in Reading 

58 . 5 % 

73 . 0 % 

67 . 6 % 

67 . 6 % 

76 . 7 % 

66 . 5 % 

Enrollment 

1787 

1738 

1392 

1131 

812 

655 

Student Daily Attendance 

70 . 8 % 

78 . 5 % 

76 . 6 % 

72 . 8 % 

77 . 7 % 

80 . 1 % 


Vaux High School 9-12 (Promise Academy) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1006.8 

1151.4 

1021.7 

1021.9 

1059.8 

1048.7 

Percent Proficient or Above 
in Math 

1 . 8 % 

15 . 8 % 

6 . 1 % 

2 . 8 % 

9 . 9 % 

8 . 0 % 

Percent Below Basic in Math 

90 . 9 % 

66 . 7 % 

81 . 7 % 

84 . 4 % 

74 . 6 % 

76 . 1 % 

PSSA Reading Average 
Scale Score 

972.8 

1074.3 

1012.3 

982.5 

1044.9 

1023.2 

Percent Proficient or Above 
in Reading 

7 . 5 % 

17 . 0 % 

13 . 3 % 

6 . 5 % 

12 . 9 % 

12 . 5 % 

Percent Below Basic in Reading 

83 . 0 % 

58 . 5 % 

74 . 7 % 

77 . 8 % 

60 . 0 % 

63 . 6 % 

Enrollment 

343 

398 

413 

455 

401 

339 

Student Daily Attendance 

78 . 2 % 

82 . 9 % 

77 . 1 % 

72 . 5 % 

76 . 3 % 

82 . 3 % 
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Guion S. Bluford / William B. Hanna School K-6 (Charter - Universal Companies) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1033.8 

1149.2 

1168.1 

1170.3 

1156.7 

1189.8 

Percent Proficient or Above 
in Math 

16 . 8 % 

22 . 4 % 

28 . 8 % 

27 . 4 % 

25 . 3 % 

27 . 2 % 

Percent Below Basic in Math 

59 . 6 % 

49 . 3 % 

44 . 9 % 

43 . 4 % 

48 . 4 % 

46 . 3 % 

PSSA Reading Average 
Scale Score 

1053.1 

1124.9 

1157.0 

1118.4 

1116.5 

1124.0 

Percent Proficient or Above 
in Reading 

17 . 7 % 

23 . 4 % 

29 . 4 % 

17 . 6 % 

17 . 3 % 

22 . 1 % 

Percent Below Basic in Reading 

61 . 4 % 

54 . 1 % 

45 . 0 % 

54 . 4 % 

57 . 4 % 

53 . 4 % 

Enrollment 

679 

711 

563 

592 

490 

554 

Student Daily Attendance 

88 . 5 % 

89 . 6 % 

91 . 3 % 

90 . 4 % 

90 . 1 % 

94 . 0 % 


Samuel Daroff School K-8 (Charter - Universal Companies) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1105.3 

1120.1 

1140.8 

1170.0 

1157.0 

1243.1 

Percent Proficient or Above 
in Math 

13 . 0 % 

14 . 5 % 

19 . 7 % 

25 . 5 % 

24 . 6 % 

40 . 6 % 

Percent Below Basic in Math 

63 . 3 % 

61 . 0 % 

56 . 3 % 

48 . 0 % 

51 . 0 % 

37 . 1 % 

PSSA Reading Average 
Scale Score 

1060.9 

1097.3 

1108.1 

1118.6 

1138.7 

1200.0 

Percent Proficient or Above 
in Reading 

16 . 3 % 

15 . 8 % 

16 . 8 % 

19 . 5 % 

23 . 5 % 

32 . 4 % 

Percent Below Basic in Reading 

61 . 9 % 

62 . 1 % 

56 . 4 % 

56 . 3 % 

51 . 9 % 

39 . 5 % 

Enrollment 

796 

842 

796 

693 

693 

715 

Student Daily Attendance 

87 . 7 % 

89 . 7 % 

90 . 3 % 

91 . 2 % 

89 . 6 % 

92 . 0 % 



Frederick Douglas School K-8 (Charter - Young Scholars) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1150.7 

1149.7 

1262.5 

1198.3 

1198.5 

1243.2 

Percent Proficient or Above 
in Math 

21 . 4 % 

19 . 8 % 

43 . 4 % 

31 . 5 % 

29 . 0 % 

43 . 5 % 

Percent Below Basic in Math 

56 . 5 % 

54 . 9 % 

31 . 6 % 

41 . 0 % 

43 . 7 % 

34 . 8 % 

PSSA Reading Average 
Scale Score 

1089.1 

1094.4 

1168.8 

1131.6 

1136.0 

1158.5 

Percent Proficient or Above 
in Reading 

18 . 9 % 

13 . 5 % 

32 . 2 % 

20 . 4 % 

21 . 7 % 

26 . 7 % 

Percent Below Basic in Reading 

60 . 5 % 

61 . 8 % 

45 . 2 % 

54 . 5 % 

52 . 0 % 

51 . 5 % 

Enrollme nt 

577 

492 

451 

478 

499 

637 

Student Daily Attendance 

88 . 7 % 

89 . 8 % 

89 . 9 % 

89 . 8 % 

90 . 2 % 

90 . 2 % 


William F. Harrity School K-8 (Charter - Mastery Charter Schools) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1162.8 

1158.3 

1185.4 

1173.1 

1218.0 

1300.3 

Percent Proficient or Above 
in Math 

34 . 9 % 

26 . 7 % 

31 . 2 % 

26 . 4 % 

36 . 8 % 

54 . 8 % 

Percent Below Basic in Math 

41 . 3 % 

42 . 9 % 

40 . 8 % 

44 . 9 % 

37 . 8 % 

24 . 9 % 

PSSA Reading Average 
Scale Score 

1076.3 

1108.5 

1120.8 

1134.0 

1150.5 

1205.8 

Percent Proficient or Above 
in Reading 

18 . 3 % 

21 . 8 % 

22 . 0 % 

22 . 3 % 

25 . 6 % 

36 . 7 % 

Percent Below Basic in Reading 

59 . 5 % 

57 . 7 % 

56 . 7 % 

52 . 1 % 

49 . 0 % 

37 . 1 % 

Enrollment 

564 

641 

537 

580 

649 

789 

Student Daily Attendance 

92 . 2 % 

92 . 1 % 

91 . 9 % 

89 . 9 % 

90 . 5 % 

92 . 0 % 
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William B. Mann School K-5 (Charter - Mastery Charter Schools) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1197.9 

1208.8 

1232.1 

1212.1 

1229.4 

1315.3 

Percent Proficient or Above 
in Math 

42 . 4 % 

34 . 8 % 

43 . 8 % 

44 . 0 % 

46 . 4 % 

59 . 0 % 

Percent Below Basic in Math 

34 . 3 % 

36 . 9 % 

30 . 4 % 

29 . 1 % 

30 . 6 % 

20 . 0 % 

PSSA Reading Average 
Scale Score 

1175.8 

1170.1 

1187.4 

1172.2 

1193.6 

1211.3 

Percent Proficient or Above 
in Reading 

35 . 3 % 

26 . 8 % 

34 . 5 % 

33 . 1 % 

39 . 9 % 

44 . 1 % 

Percent Below Basic in Reading 

34 . 3 % 

48 . 0 % 

36 . 1 % 

39 . 2 % 

36 . 6 % 

31 . 3 % 

Enrollment 

444 

429 

412 

378 

384 

429 

Student Daily Attendance 

90 . 8 % 

90 . 9 % 

91 . 0 % 

91 . 9 % 

92 . 8 % 

92 . 0 % 


Franklin Smedley School K-5 (Charter - Mastery Charter Schools) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1202.5 

1174.6 

1176.8 

1163.2 

1155.4 

1244.3 

Percent Proficient or Above 
in Math 

39 . 9 % 

34 . 9 % 

33 . 1 % 

28 . 3 % 

29 . 0 % 

44 . 9 % 

Percent Below Basic in Math 

31 . 9 % 

36 . 8 % 

47 . 2 % 

44 . 9 % 

50 . 6 % 

29 . 6 % 

PSSA Reading Average 
Scale Score 

1058.7 

1109.3 

1100.2 

1094.3 

1087.2 

1173.0 

Percent Proficient or Above 
in Reading 

22 . 1 % 

19 . 6 % 

18 . 5 % 

18 . 1 % 

18 . 9 % 

33 . 2 % 

Percent Below Basic in Reading 

59 . 2 % 

59 . 9 % 

61 . 6 % 

65 . 5 % 

63 . 8 % 

43 . 7 % 

Enrollment 

668 

630 

619 

600 

565 

607 

Student Daily Attendance 

89 . 1 % 

88 . 7 % 

88 . 9 % 

89 . 7 % 

90 . 4 % 

92 . 0 % 



John B. Stetson Middle School 5-8 (Charter - Aspira) 



2005-06 

2006-07 

2007-08 

2008-09 

2009-10 

2010-11 

PSSA Math Average 
Scale Score 

1142.7 

1125.8 

1146.2 

1169.2 

1188.3 

1284.7 

Percent Proficient or Above 
in Math 

16 . 4 % 

14 . 5 % 

19 . 4 % 

18 . 6 % 

25 . 4 % 

47 . 4 % 

Percent Below Basic in Math 

60 . 8 % 

65 . 9 % 

65 . 9 % 

56 . 4 % 

52 . 7 % 

30 . 4 % 

PSSA Reading Average 
Scale Score 

1030.4 

1024.5 

1067.1 

1086.7 

1113.6 

1155.8 

Percent Proficient or Above 
in Reading 

12 . 1 % 

8 . 5 % 

15 . 3 % 

17 . 2 % 

22 . 6 % 

28 . 2 % 

Percent Below Basic in Reading 

70 . 3 % 

74 . 8 % 

63 . 8 % 

59 . 3 % 

53 . 3 % 

46 . 1 % 

Enrollment 

810 

765 

703 

597 

629 

681 

Student Daily Attendance 

75 . 8 % 

81 . 2 % 

83 . 5 % 

86 . 6 % 

88 . 2 % 

92 . 0 % 


53 



Renaissance Charters: Comparison of Student Subgroup Enrollment 

Table B8 



% Free Lunch 
Student Enrollment 

% ELL 

Student Enrollment 

% Special Ed. 
Student Enrollment 

% Minority 
Student Enrollment 


2009-10 

2010-11 

2009-10 

2010-11 

2009-10 

2010-11 

2009-10 

2010-11 

Bluford 

95.7 

100.0 

0.4 

0.0 

14.1 

15.2 

100.0 

100.0 

Daroff 

92.9 

97.0 

0.1 

0.0 

11.3 

13.6 

99.9 

99.6 

Douglass 

94.0 

100.0 

0.0 

0.0 

13.6 

14.8 

99.6 

99.8 

Harrity 

88.4 

94.7 

0.2 

0.3 

11.1 

9.9 

99.7 

99.9 

Mann 

82.9 

89.0 

0.8 

5.1 

7.0 

9.6 

99.7 

100.0 

Smedley 

94.2 

93.9 

3.2 

3.0 

13.8 

19.8 

94.2 

94.4 

Stetson 

92.5 

99.0 

30.5 

16.4 

19.9 

19.1 

97.0 

98.2 


Figure B9 


Figure Bio 


Renaissance Charters: 

% Free Lunch Student Enrollment 


Renaissance Charters: 

% ELL Student Enrollment 



■ 2009-10 2010-11 


■ 2009-10 2010-11 


• There was a slight increase in the percentage of 
students eligible for free lunch in all but one of the 
Renaissance Charter Schools from the 2009-2010 
school year to the 2010-2011 school year. 


• There was little or no in the percentage of ELL students in 
all but two of the Renaissance Charter Schools from the 
2009-2010 school year to the 2010-2011 school year. 

* Stetson reported 112 ELL students in 2010-11 school year to RFA for this 
report, or roughly 17% of their overall student population. In the same year, 
Stetson reported 177 ELL students tested on the Math PSSA to the PA 
Department of Education, or roughly 26% of their overall student population. 
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Figure Bn 


Figure B12 


Renaissance Charters: 

% Special Education Student 
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■ 2009-10 2010-11 


Renaissance Charters: 

% Minority Student Enrollment 



■ 2009-10 2010-11 


• There was a slight increase in the percentage of 
special education students in all but two of the 
Renaissance Charter Schools from the 2009-2010 
school year to the 2010-2011 school year. 


• There was little or no change in the percentage of 
minority students in each of the Renaissance 
Charter Schools from the 2009-2010 school year 
to the 2010-2011 school year. 
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Appendix C: Quantitative Methodology 


Data and Study Population 

The population for the analyses described in this Technical Appendix includes n Renaissance Schools 30 
and a comparison group of 85 K-8 District schools that received a 7-10 on the District’s School 
Performance Index after the 2009-10 academic year. The District provided school level records, and 
grade level PSSA performance measures, for six years (2005-06^2010-11): the five years prior to the 
start of the Renaissance Schools Initiative and the first year of the Initiative itself. Renaissance Charter 
providers submitted school level records, and grade level PSSA performance, for the 2010-11 academic 
year. 

Multi-Level Modeling 

The statistical analysis for school outcomes relied on multi-level models. Student achievement 
outcomes presented in the main findings are based on a series of 3-Level models with grade levels 
nested within years (time points) nested within schools. The school attendance models are 2-Level 
models with school aggregate attendance rates nested within time. In each of the models run for these 
analyses, the outcomes were school level aggregate measures that vary each year. The analyses 
presented in the main findings of the report were generated from seven separate models, one for each of 
the seven outcomes: 

1) Math PSSA Scale Scores; 

2) Reading PSSA Scale Scores; 

3) The percentage of students scoring proficient or above on the Math PSSA; 

4) The percentage of students scoring proficient or above on the Reading PSSA; 

5) The percentage of students scoring below basic on the Math PSSA; 

6) The percentage of students scoring below basic on the Reading PSSA; 

7) Average Daily Attendance. 

The general model below represents the formal model structure that was employed for each of the six 
student achievement outcomes and represent a 3-Level nested data structure, with grade levels at Level 
1, nested within different years (time points) at Level 2, and within schools at Level 3. 

Following the formal modeling structure, is a table presenting the indicators that make up each of the 
models presented in this Appendix. 


3° The two Promise High Schools, Vaux and University City, were not included in this analysis due to the grade level 
differences and the general non-equivalence of K-8 schools and High Schools. 
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Formal Summary of the Multi-Level Models 

Level-1 Model (between grade cohorts) 

Y (Outcome) = Po*(Intercept) + Pi*(Grade 4) + P2*(Grade 5) + P3*(Grade 6) + P4*(Grade 7) + 
P5*(Grade 8) + E 


Level-2 Model (between years) 

Po = Boo + Bo l" (Year) + B02" (Post-Intervention Year) + Bo3*(Total Enrollment) + Bo4*(%FRL) + 
B05*(Student Transfers) + Bo6*(Teacher Experience) + Bo7*(% Black) + Bo8*(% Native Am.) + 
Bo 9*(% Asian) + Boio*(% Hispanic) + Bou*(% Other Ethnicity) + Boi2*(% Female) + Boi3*(% 
ELL) + Boi4*(% Spec. Ed.) + Boi5*(Student/Teacher Ratio) + Ro 
Pi = Bio 
P2 = B20 + R2 
P3 = B30 + R3 
P4 = B40 + R4 
P5 = B50 + R5 


Level-3 Model (between schools) 

Boo = Gooo + Gooi*(Elementary School) + Goo2*(Treatment) + Uoo 

B01 = G010 + Goii*(Treatment) + U01 

Bo 2 = G020 + G02i*(Treatment) + U02 

B03 = G030 

B04 = G040 

B05 = G050 

B06 = G060 

B07 = G070 

B08 = G080 

B09 = G090 

B010 = G0100 

Boil = G0110 

B012 = G0120 

B013 = G0130 

B014 = G0140 

B015 = G0150 

Bio = G100 

B20 = G200 + U20 

B30 = G300 + U30 

B40 = G400 + U40 

B50 = G500 + U50 
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Table Cl. Multi-Level Model Indicators 


School Level Outcome Measures 

Treatment 

Control Measures 

Student Achievement 

1) Average PSSA Scaled Scores - 
Math 

2) Average PSSA Scaled Scores - 
Reading 

3) % of Students Proficient or 
Above - Math 

4) % of Students Proficient or 
Above - Reading 

5) % of Students Below Basic - 
Math 

6 ) % of Student Below Basic - 
Reading 

Attendance 

7) School Average Daily 
Attendance 

Participation in the 
Renaissance Schools 
Initiative 

Time 

• Pre-Intervention (2005-06 -> 
2009-10) 

Comparison Group 

• Elementary and Middle 
schools with an SPI of 7-10 in 
the 2009-10 academic year 

Grade Level 

• 3 rd grade as the comparison 
group 

School Level Controls 

• Elementary v. Middle School 

• Socio-demographics 

0 % ELL, Special Ed., 
Free/Reduced Lunch, 
% Minority 

• Enrollment 

• Student Retentions 1 

• Average Teacher Years at 
current school 

• Student :Teacher Ratio 32 


Model Specification & Review 

At Level l, Po represents aggregate 3 rd grade performance along each outcome measure, and coefficient 
for Pi - P5 represent the difference between each grade level’s performance (grades 4-8) and 3 rd grade 
performance along each of the different PSSA measures. In each model, each grade was modeled as a 
separate dummy variable as growth patterns varied by specific grade level. 

Variation was analyzed between treatment impacts and each grade level, however no consistent 
patterns emerged. While some differences seemed to exist superficially these were neither consistent 
nor statistically significant. Given the low number of treatment schools in the sample, the standard 
error of the estimates is large, particularly when measuring interactions between multiple factors. A 
larger sample size may reveal differences in the treatment effect by specific grade levels that are 
statistically significant, but, at the same time, any effects could also disappear with more schools if any 
observed variation had been driven by individual schools in this small sample. Another factor that may 
explain the lack of ‘grade level’ effects is the idiosyncratic qualities inherent in each individual cohort of 
students each year in each grade at a given school. For the purposes of these analyses, there was no 
clear variation in treatment effects by grade level. 


3 1 “Student Retention” was calculated as the percentage of students who attended a single school in consecutive years, i.e. for 
the 2010-11 school year a school’s ‘student mobility’ would be calculated at the percentage of students in 2010-11 who also 
attended the same school in 2009-10. 

3 2 “Teachers” were reported as the number of full time equivalent teachers at each school for each year. 
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At Level 2, Boo represents schools’ initial and aggregated outcome levels in the 2005-06 school year, 
having already controlled for the grade level variation at Level 1. B01 represents natural yearly growth 
rates in a schools aggregate outcome levels over time. B02 in these models represents any overall 
change in outcome levels/or all schools after the implementation of the Renaissance Schools Initiative 
in the school district in the 2010-11 school year. B03-015 represents a vector of the moderating 
variables and school characteristics that vary from year to year to control for variation across these 
indicators among the schools in both the study and comparison populations. 

At level 3, G001 represents a control for whether a school was an elementary or a middle school and 
G002 represents the treatment group differences in initial outcome levels in 2005-06. G011 represents 
treatment group differences on growth rate over time, and G021 represents the treatment impact on 
outcomes in the 2010-11 school year. Uoo through U02 represent the unique variations between 
schools in initial outcomes levels, change over time, and change in 2010-11 specifically. 

Growth along each outcome was consistent by individual years, and so time (school year) was measured 
as a continuous variable, with the addition of a dummy variable representing 2010-11, or the post- 
intervention time-point. All other control measures were grand mean centered, thus results of the 
models are for schools with the sample averages of those measures. The relations between time and 
grade levels and the outcomes were allowed to vary randomly by year and school, but the relationships 
between the demographic and contextual factors and outcomes were fixed as they did not have 
significant or consistent variation. This also allows for greater ease and consistency in reporting. 

Below are the estimates from each of the seven models that were run for the analyses in this report, 
measuring treatment as participation in the Renaissance Schools Initiative in comparison to the group 
SPI 7-10 schools. 

In addition to the seven models presented here, seven additional models were run in which the 
Renaissance Schools group was split into Renaissance Charters and Promise Academies to assess any 
statistically significant differences in the performance of Renaissance Charters and Promise Academies 
along each outcome measure. These models showed no significant differences between Renaissance 
Charters and Promise Academies, and the results of these models follow the seven main models run for 
the analyses in the interim report. 

T reatment Effect Sizes 

The tables on the following pages present the treatment differences along each outcome over time; 
these differences were extracted from the output of each individual model. There are three tables, one 
for all Renaissance schools combined, a second for Charter schools, and a third for Promise Academies 
(although the estimates of the latter two are shown separately, the results were derived from the same 
model). While the effect sizes presented in each of the tables below are substantial and represent 
significant change in the treatment group of schools in Year One of the Renaissance Schools Initiative, 
it is important to consider the historical context at the study schools and within the District to fully 
assess the overall impact of the intervention. 33 


33 Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. 
Child Development Perspectives, 2, 172-177. 
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The effect size columns present the treatment effects in terms of how large a shift they represent in the 
sample population, in the measure of standard deviation units. The column “Effect Size” does this using 
the sample distribution and standard deviations at the individual grade and cohort levels for each 
outcome, while the column “School Level Effect Size” does this using the sample distribution and 
standard deviations at the aggregated school level for each outcome. The first represents the effect size 
in terms of the ‘unit of observation’ as the outcomes are observed and measured at the level of each 
individual grade and cohort, and this is the typical way in which effect sizes are reported. However, the 
latter represents the effect size in terms of the ‘unit of treatment’, and is the more relevant from a 
theoretical point of view to the questions raised in this study. The latter are also larger as it is more 
difficult to raise an entire school’s average outcome by “X” amount than it is to raise an individual 
grade’s or cohort’s. However, for this specific set of results, both sets of effect sizes are substantial and 
educationally relevant. 

Table C2. Student Outcomes: Renaissance Schools v. Comparison Schools 



* Indicates that group differences are statistically significant, p<.05 


34 Initial Level Difference represents the difference between the Renaissance School Averages and the Comparison School 
Averages after controlling for the following cohort and school level factors: grade level, school grade configuration, school 
enrollment, student socio-demographics(percent Minority, ELL, Special Ed., Free Lunch), student retention, average teacher 
years at school, and student : teacher ratio. 

35 Growth Rate Difference represents the difference between the Renaissance Schools’ average rate of growth and the 
Comparison School average rate of growth for each outcome from 2005-06 through 2010-11, controlling for all the other 
indicators included in each model. 

3 6 Post Intervention Difference represents the difference between the Renaissance Schools’ average change for each outcome 
and the Comparison Schools’ average change for each outcome in Year One of the Renaissance Schools Initiative, controlling 
for all the other indicators included in each model. 

37 Effect Size represents the standard deviation unit shift in the overall sample population as a result of the treatment at the 
level of each individual grade and cohort. 

38 School Level Effect Size represents the standard deviation unit shift in the overall sample population as a result of the 
treatment at the school level. School Level Effect Sizes are much larger because the variation is much smaller at the school level 
due to fewer numbers of schools in the overall sample. 
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Table C3. Student Outcomes: Renaissance Charters v. Comparison Schools A 


Initial 

Outcome Level 39 Growth 

Measure (2005- Rate* 0 

06) 

p School 

T os . Effect Level 

7 ™.'!', Size* 2 Effect 

f 2010 ' 111 Size*'. 

Math Scale Score -42.2* -2.2 

83.7* 1.13 2.24 

Math % 

Proficient or -7.1* -0.7 

Above 

15.7* .98 1.88 

Math % Below * , 

0 • 7.7 0.6 

Basic 

-14.2* -.88 -1.78 

Reading Scale * 

Score - 34 ' 7 

38.0* .49 1.06 

Reading % 

Proficient or -7.3* -0.7 

Above 

8.4* -57 1.17 

Reading % Below * 

Basic 77 02 

-10.2* -.66 -1.32 

Attendance 

Rates - 1 ’ 41 °’ 17 

1.22 .63 .79 


* Indicates that group differences are statistically significant, p<.05 


39 Initial Level Difference represents the difference between the Renaissance School Averages and the Comparison School 
Averages after controlling for the following cohort and school level factors: grade level, school grade configuration, school 
enrollment, student socio-demographics(percent Minority, ELL, Special Ed., Free Lunch), student retention, average teacher 
years at school, and student : teacher ratio. 

40 Growth Rate Difference represents the difference between the Renaissance Schools’ average rate of growth and the 
Comparison School average rate of growth for each outcome from 2005-06 through 2010-11, controlling for all the other 
indicators included in each model. 

41 Post Intervention Difference represents the difference between the Renaissance Schools’ average change for each outcome 
and the Comparison Schools’ average change for each outcome in Year One of the Renaissance Schools Initiative, controlling 
for all the other indicators included in each model. 

42 Effect Size represents the standard deviation unit shift in the overall sample population as a result of the treatment at the 
level of each individual grade and cohort. 

43 School Level Effect Size represents the standard deviation unit shift in the overall sample population as a result of the 
treatment at the school level. School Level Effect Sizes are much larger because the variation is much smaller at the school level 
due to fewer numbers of schools in the overall sample. 


61 




Table C4. Student Outcomes: Promise Academies v. Comparison Schools A 



* Indicates that group differences are statistically significant, p<.05 


44 Initial Level Difference represents the difference between the Renaissance School Averages and the Comparison School 
Averages after controlling for the following cohort and school level factors: grade level, school grade configuration, school 
enrollment, student socio-demographics(percent Minority, ELL, Special Ed., Free Lunch), student retention, average teacher 
years at school, and student : teacher ratio. 

45 Growth Rate Difference represents the difference between the Renaissance Schools’ average rate of growth and the 
Comparison School average rate of growth for each outcome from 2005-06 through 2010-11, controlling for all the other 
indicators included in each model. 

46 Post Intervention Difference represents the difference between the Renaissance Schools’ average change for each outcome 
and the Comparison Schools’ average change for each outcome in Year One of the Renaissance Schools Initiative, controlling 
for all the other indicators included in each model. 

4 “ Effect Size represents the standard deviation unit shift in the overall sample population as a result of the treatment at the 
level of each individual grade and cohort. 

48 School Level Effect Size represents the standard deviation unit shift in the overall sample population as a result of the 
treatment at the school level. School Level Effect Sizes are much larger because the variation is much smaller at the school level 
due to fewer numbers of schools in the overall sample. 


62 




Table C5. Student Outcomes: Promise Academies v. Renaissance Charters A 


Initial 

Outcome Measure Level 

(2005- 

06)49 

Post- 

Intervention 

(2010-11)5° 

Math Scale Score -10.49 

-1.21 

Math % Proficient or 0 

Above ' 2 ' 82 

4.62 

Math % Below Basic 3.34 

- 7-99 

Reading Scale Score -5.90 

49.82 

Reading % Proficient or 
Above °'°9 

5-14 

Reading % Below Basic 1.37 

-12.77 

Attendance Rates 1.20 

0.69 

* Indicates that group differences are statistical 

y significant, p<.05 


49 Initial Level Difference represents the difference between the Renaissance School Averages and the Renaissance Charter 
Averages after controlling for the following cohort and school level factors: grade level, school grade configuration, school 
enrollment, student socio-demographics(percent Minority, ELL, Special Ed., Free Lunch), student retention, average teacher 
years at school, and student : teacher ratio. 

s° Post Intervention Difference represents the difference between the Renaissance Schools’ average change for each outcome 
and the Renaissance Charter Schools’ average change for each outcome in Year One of the Renaissance Initiative, controlling 
for all the other indicators included in each model. 
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Multi-Level Model Results 


Model I: Math Scale Score - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1158.91 

94-954 

.000*** 

Elementary Schools 
Prior Level 

-7-53 

-0.710 

.480 

Renaissance 
Prior Level 

-46.29 

-3-796 

. 000 *** 

Time/Yearly Growth 

9-77 

5-083 

. 000 *** 

Renaissance 
Time/Yearly Growth 

-2-45 

-0.870 

•387 

Post-Intervention 

-9.42 

-2.114 

•037* 

Renaissance 

Post-Intervention 

82.48 

5-157 

. 000 *** 

Total Enrollment 

-0.01 

-0-135 

•893 

% FRL 

-0.38 

-1.663 

•097 

Student Transfers 

-0.08 

-1.245 

.214 

Teacher Experience 

3-24 

3.028 

. 003 ** 

% Black 

-0.58 

-0.834 

•405 

% Native 

13-38 

1.708 

.088 

% Asian 

1.71 

1-514 

.130 

% Hispanic 

-0.06 

-0.085 

•932 

% Other Ethnicity 

-0.40 

-0.183 

•855 

% Female 

0.58 

0.530 

•596 

% ESL 

-2.27 

-3-485 

.001*** 

% Spec. Ed. 

-2.77 

-3.224 

. 002 ** 

Student-Teacher Ratio 

-5-72 

-4.141 

. 000 *** 

Grade 4 

61.50 

20.265 

.000*** 

Grade 5 

65-31 

13.018 

. 000 *** 

Grade 6 

70.96 

14.736 

. 000 *** 

Grade 7 

62.88 

8.871 

. 000 *** 

Grade 8 

48.25 

8.688 

.000*** 

Proficient or Advanced - All Renaissance Schools v. Comparison Set 

Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

44-98 

18.822 

.000*** 

Elementary Schools 
Prior Level 

-2.60 

-1.283 

.203 

Renaissance 
Prior Level 

-8.22 

-4-139 

.000*** 

Time/Yearly Growth 

2.24 

5-523 

. 000 *** 

Renaissance 
Time/Yearly Growth 

-1.08 

-1-833 

.070 

Post-Intervention 

-1.76 

-1.668 

•099 

Renaissance 

Post-Intervention 

17.88 

5-312 

. 000 *** 
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Total Enrollment 

-0.01 

-0.320 

•749 

% FRL 

-0.10 

-1.524 

.128 

Student Transfers 

-0.01 

-0.932 

•352 

Teacher Experience 

0.60 

2.526 

.012* 

% Black 

-0.19 

-1.837 

.066 

% Native 

3-61 

2-355 

.019* 

% Asian 

0.29 

1-391 

.165 

% Hispanic 

-0.08 

-0.700 

.484 

% Other Ethnicity 

-0.52 

- 1-173 

.242 

% Female 

0.23 

0-943 

•347 

% ESL 

-0.50 

-3-886 

.000*** 

% Spec. Ed. 

-0.58 

- 3-147 

.002** 

Student-Teacher Ratio 

- 1-35 

- 4-500 

.000*** 

Grade 4 

-2.99 

-4.292 

.000*** 

Grade 5 

- 14-05 

- 13-341 

.000*** 

Grade 6 

-11.84 

-10.861 

.000*** 

Grade 7 

- 13-56 

-11.274 

.000*** 

Grade 8 

-14.20 

-12.104 

.000*** 
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Model III: Math % Below Basic - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

27.96 

11.677 

.000*** 

Elementary Schools 
Prior Level 

0.85 

0.372 

.710 

Renaissance 
Prior Level 

8-75 

4-145 

. 000 *** 

Time/Yearly Growth 

-2.14 

-5.871 

. 000 *** 

Renaissance 


1.288 


Time/Yearly Growth 

0.71 

.202 

Post-Intervention 

3-68 

4.011 

. 000 *** 

Renaissance 

Post-Intervention 

-18.13 

-5-067 

. 000 *** 

Total Enrollment 

0.01 

0.288 

•773 

% FRL 

0.05 

0-775 

•439 

Student Transfers 

0.02 

1.069 

.286 

Teacher Experience 

-0.72 

- 3-449 

.001*** 

% Black 

0.12 

0.800 

•424 

% Native 

-2.61 

-1.688 

.092 

% Asian 

-0.23 

-0.951 

•343 

% Hispanic 

0.03 

0.204 

•839 

% Other Ethnicity 

0.13 

0.292 

•77 1 

% Female 

-0.08 

-0.346 

•729 

% ESL 

0.51 

4.811 

.000*** 

% Spec. Ed. 

0.61 

3-659 

.001*** 

Student-Teacher Ratio 

1.15 

4.207 

.000*** 

Grade 4 

13-19 

22.022 

.000*** 

Grade 5 

13.70 

15-077 

.000*** 

Grade 6 

15-56 

16.100 

.000*** 

Grade 7 

21.04 

13-372 

.000*** 

Grade 8 

20.37 

14-795 

.000*** 
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Model IV: Reading Scale Score - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1154-67 

116.866 

.000*** 

Elementary Schools 
Prior Level 

7.82 

0.845 

.401 

Renaissance 
Prior Level 

- 36.59 

-4.048 

.001*** 

Time/Yearly Growth 

14-55 

8.991 

. 000 *** 

Renaissance 
Time/Yearly Growth 

-2.31 

-0.918 

.362 

Post-Intervention 

-16.51 

- 3-864 

. 000 *** 

Renaissance 

Post-Intervention 

64.19 

4.386 

. 000 *** 

Total Enrollment 

-0.01 

-0.368 

•713 

% FRL 

-0.67 

-2.177 

.030* 

Student Transfers 

-0.08 

- 1-193 

•234 

Teacher Experience 

3-63 

3-705 

.000*** 

% Black 

-0.40 

-0.583 

.560 

% Native 

13.07 

2.124 

.034* 

% Asian 

0.93 

0.932 

•352 

% Hispanic 

-0.33 

-0.442 

.658 

% Other Ethnicity 

-1.41 

-0.671 

.502 

% Female 

1.92 

1-937 

•053 

% ESL 

-1.04 

- 1-731 

.084 

% Spec. Ed. 

-2.11 

-3-063 

. 003 ** 

Student-Teacher Ratio 

-I.67 

-1.386 

.166 

Grade 4 

- 39-41 

- 13-451 

. 000 *** 

Grade 5 

- 79-69 

-23.071 

. 000 *** 

Grade 6 

- 41-54 

- 10-343 

. 000 *** 

Grade 7 

6-73 

1.403 

.164 

Grade 8 

52.20 

9.223 

.000*** 
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Model V: Reading % Proficient or Advanced - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

32.15 

15459 

.000*** 

Elementary Schools 
Prior Level 

1.28 

0.723 

472 

Renaissance 
Prior Level 

-7.18 

-4.962 

. 000 *** 

Time/Yearly Growth 

2-35 

7.242 

. 000 *** 

Renaissance 
Time/Yearly Growth 

- 0-93 

-1.897 

.061 

Post-Intervention 

-341 

-4.095 

. 000 *** 

Renaissance 

Post-Intervention 

11.05 

4.380 

. 000 *** 

Total Enrollment 

0.01 

0.880 

.380 

% FRL 

-0.14 

-2.086 

.037* 

Student Transfers 

-0.03 

- 2.133 

•033* 

Teacher Experience 

0-53 

2.787 

.006** 

% Black 

-0.12 

-1.031 

•304 

% Native 

3-13 

2.323 

.021* 

% Asian 

0.23 

1.190 

•235 

% Hispanic 

-0.08 

-0.634 

.526 

% Other Ethnicity 

-0.42 

-0.968 

•334 

% Female 

0-35 

1.709 

.088 

% ESL 

-0.29 

- 2.139 

• 033 * 

% Spec. Ed. 

- 0-37 

-2.625 

.009** 

Student-Teacher Ratio 

- 0-54 

-2.212 

.027* 

Grade 4 

-7.04 

- 9-550 

. 000 *** 

Grade 5 

-14.99 

-19493 

. 000 *** 

Grade 6 

-12.23 

-12.360 

. 000 *** 

Grade 7 

- 5-01 

- 4-330 

. 000 *** 

Grade 8 

5-76 

4-777 

. 000 *** 
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Model VI: Reading % Below Basic - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

48.46 

23-825 

.000*** 

Elementary Schools 
Prior Level 

-1.44 

-0.780 

•438 

Renaissance 
Prior Level 

8.09 

3-501 

.001*** 

Time/Yearly Growth 

-2.29 

-6.781 

. 000 *** 

Renaissance 
Time/Yearly Growth 

0.50 

0.746 

•458 

Post-Intervention 

2.88 

3-312 

. 002 *** 

Renaissance 

Post-Intervention 

-16.72 

-4.152 

. 000 *** 

Total Enrollment 

0.01 

0-559 

•576 

% FRL 

0.17 

2.400 

.017* 

Student Transfers 

0.01 

0.772 

•440 

Teacher Experience 

-0.89 

-4.160 

.000*** 

% Black 

-0.03 

-0.215 

.830 

% Native 

- 4-15 

-2.987 

.003** 

% Asian 

- 0-35 

- 1-539 

.124 

% Hispanic 

-0.03 

- 0-153 

•879 

% Other Ethnicity 

-0.10 

-0.225 

.822 

% Female 

- 0.49 

-2.605 

.010** 

% ESL 

0.25 

1.712 

.087 

% Spec. Ed. 

0.42 

3.017 

. 003 ** 

Student-Teacher Ratio 

0.46 

1-975 

.048* 

Grade 4 

1.91 

2.729 

.007** 

Grade 5 

12.18 

14.966 

. 000 *** 

Grade 6 

3-82 

3-557 

.001*** 

Grade 7 

- 3-69 

-1.825 

.071 

Grade 8 

- 7-53 

-5.082 

.000*** 
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Model VII: Attendance Rates - All Renaissance Schools v. Comparison Schools 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

88.24 

198.638 

.000*** 

Elementary Schools 
Prior Level 

1.85 

4.184 

. 000 *** 

Renaissance 
Prior Level 

-0.97 

-1.096 

.277 

Time/Yearly Growth 

0-45 

9-334 

. 000 *** 

Renaissance 
Time/Yearly Growth 

-0.02 

-0.086 

•932 

Post-Intervention 

- 0-53 

- 3-849 

. 000 *** 

Renaissance 

Post-Intervention 

1.49 

2.998 

.004** 

Total Enrollment 

-0.01 

-1.670 

•095 

% FRL 

-0.01 

- 0.945 

•345 

Student Transfers 

-0.01 

-1.912 

.056 

Teacher Experience 

0.04 

0.991 

.322 

% Black 

0.01 

0.196 

•845 

% Native 

0.11 

0.289 

.772 

% Asian 

0.11 

3-054 

. 003 ** 

% Hispanic 

0.01 

0.859 

•391 

% Other Ethnicity 

-0.01 

-0.126 

.900 

% Female 

0.01 

0.148 

.883 

% ESL 

-0.03 

-1.855 

.064 

% Spec. Ed. 

-0.04 

-2.036 

.042* 

Student-Teacher Ratio 

0.02 

0.391 

.696 
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Additional Model Results 

The model results on the following pages present models developed that separated the Renaissance 
Schools group into two separate groups: Promise Academies and Renaissance Charters. Results from 
these models were pulled for the creation of the Student Outcomes Tables presented earlier in this 
section. While the models to follow do reveal significant differences between the Promise Academies 
and the Comparison Schools in addition to significant differences between the Renaissance Charters 
and the Comparison Schools across most of the student achievement and attendance outcomes, an 
additional set of models were run to assess the significance of differences in Promise Academy and 
Renaissance Charter performance. These models changed the reference group to ‘Renaissance Charters’ 
and revealed there to be no significant differences between the Promise Academies and this comparison 
group in the post-intervention measure for each outcome. 
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Multi-Level Model Results: Promise Academies & Renaissance Charters v. Comparison 
Schools 


Model la. Math Scale Score -Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1158.73 

94.171 

.000*** 

Elementary Schools 
Prior Level 

- 7-23 

-0.672 

•503 

Charter Schools 
Prior Level 

-42.17 

-2.628 

.011* 

Promise Academies 
Prior Level 

-52.66 

- 3-779 

. 000 *** 

Time/Yearly Growth 

9-74 

5-065 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

-2.17 

-0.602 

•548 

Promise Academies 
Time/Yearly Growth 

- 3-23 

-1.015 

•313 

Post- Intervention 

-9.44 

-2.107 

.038* 

Charter Schools 
Post-Intervention 

83-72 

4-531 

. 000 *** 

Promise Academies 
Post-Intervention 

82.51 

3-308 

.002** 

Total Enrollment 

-0.01 

-0.200 

.841 

% FRL 

-0.38 

-1.712 

.087 

Student Transfers 

-0.07 

- 1-133 

.258 

Teacher Experience 

3-31 

3.020 

003** 

% Black 

- 0-57 

-0.832 

.406 

% Native 

13-23 

1.684 

.092 

% Asian 

1-73 

1-525 

.128 

% Hispanic 

-0.05 

-0.061 

•951 

% Other Ethnicity 

-0.46 

-0.209 

•834 

% Female 

0.61 

0.584 

•584 

% ESL 

-2.25 

- 3-405 

.001*** 

% Spec. Ed. 

- 2-73 

-3.190 

. 002 *** 

Student-Teacher Ratio 

- 5-79 

- 4-175 

.000*** 

Grade 4 

61.50 

20.264 

.000*** 

Grade 5 

65-31 

13.016 

.000*** 

Grade 6 

70-97 

14-738 

. 000 *** 

Grade 7 

65-77 

9.214 

.000*** 

Grade 8 

48.38 

8.713 

.000*** 
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Model Ha. Math % Proficient or Advanced - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

44-99 

18.808 

.000*** 

Elementary Schools 
Prior Level 

- 2-55 

- 1-257 

.213 

Charter Schools 
Prior Level 

-7.08 

-3.020 

.004** 

Promise Academies 
Prior Level 

-9.90 

-3-856 

. 000 *** 

Time/Yearly Growth 

2.21 

5.462 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

-0.71 

-1.044 

.300 

Promise Academies 
Time/Yearly Growth 

-1.71 

-2.172 

.033* 

Post-Intervention 

- 1-73 

- 1-633 

.106 

Charter Schools 
Post-Intervention 

15-72 

4.261 

. 000 *** 

Promise Academies 
Post-Intervention 

20.34 

3.602 

.001*** 

Total Enrollment 

-0.01 

-0.151 

.880 

% FRL 

-0.10 

-1.652 

.099 

Student Transfers 

-0.02 

-1.293 

.197 

Teacher Experience 

0-59 

2.505 

.013* 

% Black 

-0.19 

-1.852 

.064 

% Native 

3-49 

2.282 

.023* 

% Asian 

0.29 

1.409 

.160 

% Hispanic 

-0.07 

-0.642 

•521 

% Other Ethnicity 

- 0-53 

-1.212 

.227 

% Female 

0.23 

0.928 

•354 

% ESL 

-0.50 

-3-856 

.000*** 

% Spec. Ed. 

- 0-55 

- 3-030 

.003** 

Student-Teacher Ratio 

-1.36 

- 4-556 

.000*** 

Grade 4 

-3-00 

-4.292 

.000*** 

Grade 5 

-14.04 

- 13-334 

.000*** 

Grade 6 

-11.83 

-10.867 

.000*** 

Grade 7 

- 13-54 

-H -351 

.000*** 

Grade 8 

-14.17 

-12.167 

.000*** 
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Model Ilia. Math % Below Basic - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

27.84 

11.511 

.000*** 

Elementary Schools 
Prior Level 

0.87 

0-379 

-705 

Charter Schools 
Prior Level 

7-65 

3.060 

.003** 

Promise Academies 
Prior Level 

10.99 

3.622 

.001*** 

Time/Yearly Growth 

-2.11 

- 5-737 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

0-57 

0.897 

•373 

Promise Academies 
Time/Yearly Growth 

0.80 

1.059 

•293 

Post-Intervention 

3-62 

3-923 

.000*** 

Charter Schools 
Post-Intervention 

-14.15 

- 3-965 

.000*** 

Promise Academies 
Post-Intervention 

-22.14 

- 3-458 

.001*** 

Total Enrollment 

-0.01 

-0.130 

•897 

% FRL 

0.05 

0-994 

.321 

Student Transfers 

0.02 

1-705 

.088 

Teacher Experience 

-0.70 

- 3-315 

.001*** 

% Black 

0.11 

O.729 

•467 

% Native 

-2.56 

-1.681 

•093 

% Asian 

-0.22 

-O.919 

•359 

% Hispanic 

0.02 

0.120 

•905 

% Other Ethnicity 

0.12 

O.273 

•785 

% Female 

-0.07 

-O.323 

•747 

% ESL 

0.51 

4.789 

.000*** 

% Spec. Ed. 

0.61 

3-697 

.000*** 

Student-Teacher Ratio 

1.15 

4-143 

.000*** 

Grade 4 

13-19 

22.020 

.000*** 

Grade 5 

13-71 

15-053 

.000*** 

Grade 6 

15-57 

16.128 

.000*** 

Grade 7 

21.07 

13-363 

.000*** 

Grade 8 

20.40 

14-805 

.000*** 
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Model IVa. Reading Scale Score - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1155-26 

116.204 

.000*** 

Elementary Schools 
Prior Level 

7-94 

0.854 

•396 

Charter Schools 
Prior Level 

- 34-68 

-3.017 

.004** 

Promise Academies 
Prior Level 

-40.58 

-4.090 

. 000 *** 

Time/Yearly Growth 

14.27 

8.824 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

-0.96 

-0.365 

.716 

Promise Academies 
Time/Yearly Growth 

- 3-44 

-0.917 

.362 

Post-Intervention 

- 15-97 

- 3-697 

.001*** 

Charter Schools 
Post-Intervention 

38.01 

2-355 

.021* 

Promise Academies 
Post-Intervention 

87.90 

3-795 

. 000 *** 

Total Enrollment 

0.01 

0.368 

•713 

% FRL 

-0.72 

-2.785 

.006** 

Student Transfers 

-0.14 

-2.095 

.036* 

Teacher Experience 

3-43 

3-563 

.001*** 

% Black 

-0.36 

-0.526 

•599 

% Native 

12.22 

1.986 

• 047 * 

% Asian 

0.88 

0.894 

•372 

% Hispanic 

-0.26 

- 0-355 

•723 

% Other Ethnicity 

- 1-33 

-0.615 

•539 

% Female 

1.81 

1.822 

.069 

% ESL 

-1.10 

- 1-945 

.052 

% Spec. Ed. 

-2.08 

- 3-057 

003** 

Student-Teacher Ratio 

-1.52 

-1.248 

.213 

Grade 4 

- 39-41 

- 13-450 

.000*** 

Grade 5 

-79.68 

-23.042 

. 000 *** 

Grade 6 

- 41-56 

-10.352 

. 000 *** 

Grade 7 

6.68 

1.390 

.168 

Grade 8 

52.23 

9.204 

.000*** 
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Model Va. Reading % Proficient or Advanced - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

32.19 

15-449 

.000*** 

Elementary Schools 
Prior Level 

1.31 

0-734 

465 

Charter Schools 
Prior Level 

- 7-25 

- 5-316 

. 000 *** 

Promise Academies 
Prior Level 

- 7-15 

-2.719 

.■ 008 ** 

Time/Yearly Growth 

2.32 

7.174 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

-0.65 

- 1-454 

.150 

Promise Academies 
Time/Yearly Growth 

-1.28 

-1.510 

•135 

Post-Intervention 

- 3-37 

-4.014 

.000*** 

Charter Schools 
Post-Intervention 

8-43 

2.750 

.■ 008 ** 

Promise Academies 
Post-Intervention 

13-58 

3460 

.001*** 

Total Enrollment 

0.01 

1.206 

.229 

% FRL 

-0.15 

-2.316 

.021* 

Student Transfers 

-0.03 

-2.271 

.024* 

Teacher Experience 

0.51 

2.699 

.008** 

% Black 

-0.12 

-1.011 

•313 

% Native 

3 - 0 i 

2.225 

.026* 

% Asian 

0.23 

1.192 

•234 

% Hispanic 

-0.08 

-0.591 

•554 

% Other Ethnicity 

-0.41 

-0.938 

•349 

% Female 

0-34 

1.663 

•097 

% ESL 

-0.30 

-2.247 

.025* 

% Spec. Ed. 

-0.36 

-2.596 

.010** 

Student-Teacher Ratio 

- 0-53 

- 2.155 

.031* 

Grade 4 

-7.04 

- 9-550 

.000*** 

Grade 5 

-14.99 

-19477 

. 000 *** 

Grade 6 

12.23 

-12.368 

. 000 *** 

Grade 7 

-5-02 

4-337 

. 000 *** 

Grade 8 

5-76 

4.778 

. 000 *** 
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Model Via. Reading % Below Basic - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

48.35 

23-453 

.000*** 

Elementary Schools 
Prior Level 

- 1-45 

- 0-774 

•441 

Charter Schools 
Prior Level 

7.69 

2.389 

.019* 

Promise Academies 
Prior Level 

9.06 

4-597 

. 000 *** 

Time/Yearly Growth 

-2.25 

-6.718 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

0.23 

0.309 

-758 

Promise Academies 
Time/Yearly Growth 

0-73 

0.710 

.480 

Post-Intervention 

2-75 

3-119 

.003*** 

Charter Schools 
Post-Intervention 

-10.19 

-2.669 

.010** 

Promise Academies 
Post-Intervention 

-22.97 

-3-186 

.002** 

Total Enrollment 

-0.01 

-0.220 

.826 

% FRL 

0.18 

3-409 

.001*** 

Student Transfers 

0.03 

1.742 

.082 

Teacher Experience 

-0.84 

- 3-895 

.000*** 

% Black 

-0.05 

-0.294 

.769 

% Native 

- 3-98 

-2.791 

.006** 

% Asian 

- 0-34 

- 1-517 

.130 

% Hispanic 

-0.04 

-0.261 

•794 

% Other Ethnicity 

-0.10 

-0.226 

.822 

% Female 

-0.46 

- 2-454 

.015* 

% ESL 

0.27 

1-933 

•053 

% Spec. Ed. 

0.42 

3.060 

. 003 ** 

Student-Teacher Ratio 

0-43 

1.791 

•073 

Grade 4 

1.91 

2.728 

.007** 

Grade 5 

12.18 

14-953 

.000*** 

Grade 6 

3-82 

3-543 

.001*** 

Grade 7 

- 3-58 

- 1-759 

.082 

Grade 8 

-7.40 

-4.889 

.000*** 
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Model Vila. Attendance Rates - Comparison Schools Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

88.25 

196.575 

.000*** 

Elementary Schools 
Prior Level 

1.86 

4.188 

. 000 *** 

Charter Schools 
Prior Level 

-1.41 

-1.065 

.291 

Promise Academies 
Prior Level 

-0.20 

-0.313 

-755 

Time/Yearly Growth 

0-45 

9.268 

. 000 *** 

Charter Schools 
Time/Yearly Growth 

0.17 

0.461 

.646 

Promise Academies 
Time/Yearly Growth 

-0.36 

-1.781 

.078 

Post-Intervention 

- 0-53 

-3-846 

. 000 *** 

Charter Schools 
Post-Intervention 

1.22 

1.764 

.081 

Promise Academies 
Post-Intervention 

1.91 

2.874 

. 006 ** 

Total Enrolment 

-0.01 

- 1-635 

.102 

% FRL 

-0.01 

-0.896 

•371 

Student Transfers 

-0.01 

- 1-931 

•054 

Teacher Experience 

0.04 

1.011 

•313 

% Black 

0.01 

0.228 

.820 

% Native 

0.10 

0.241 

.810 

% Asian 

0.11 

3-390 

. 003 ** 

% Hispanic 

0.01 

0.928 

•354 

% Other Ethnicity 

-0.01 

-0.149 

.882 

% Female 

0.01 

0.242 

.809 

% ESL 

-0.03 

-1.740 

.082 

% Spec. Ed. 

-0.04 

-1.996 

.046* 

Student-Teacher Ratio 

0.01 

0-339 

•734 
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Model Results: Promise Academies & Comparison Schools vs. Renaissance Charters 

* Highlighting throughout this section indicates non-significant differences between Promise Academies and Renaissance 
Charters 


Math Scale Score - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1116.56 

56.218 

.000*** 

Elementary Schools 
Prior Level 

- 7-23 

-0.614 

•541 

Promise Academies 
Prior Level 

-10.49 

-0.383 

•703 

SPI Comparison Schools 
Prior Level 

42.17 

2.388 

.019* 

Time/Yearly Growth 

7-57 

1.508 

•135 

Promise Academies 
Time/Yearly Growth 

-1.06 

-0.131 

•897 

SPI Comparison Schools 
Time/Yearly Growth 

2.17 

0.419 

.676 

Post-Intervention 

74.28 

3-763 

.001*** 

Promise Academies 
Post-Intervention 

-1.21 

-0.045 

•965 

SPI Comparison Schools 
Post-Intervention 

-83.72 

-4.064 

. 000 ** 

Total Enrolment 

-O.Ol 

-0.200 

.841 

% FRL 

-0.38 

-1.712 

.087 

Student Transfers 

-0.07 

- 1-133 

.258 

Teacher Experience 

3-31 

3.020 

003** 

% Black 

- 0-57 

-0.832 

.406 

% Native 

13-23 

1.684 

.092 

% Asian 

1-73 

1-525 

.128 

% Hispanic 

-0.05 

-0.061 

•951 

% Other Ethnicity 

-0.46 

-0.209 

.834 

% Female 

0.61 

0.584 

•584 

% ESL 

-2.25 

-3.405 

.001*** 

% Spec. Ed. 

- 2-73 

-3.190 

.002*** 

Student-Teacher Ratio 

- 5-79 

- 4-175 

. 000 *** 

Grade 4 

61.50 

20.264 

. 000 *** 

Grade 5 

65-31 

13.016 

. 000 *** 

Grade 6 

70.97 

14.738 

.000*** 

Grade 7 

65-77 

9.214 

. 000 *** 

Grade 8 

48.38 

8.713 

. 000 *** 
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Math % Proficient or Advanced - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

37-92 

13.096 

.000*** 

Elementary Schools 
Prior Level 

- 2-55 

- 1-257 

.213 

Promise Academies 
Prior Level 

-2.82 

-0.899 

•372 

SPI Comparison Schools 
Prior Level 

7.08 

3.020 

.004** 

Time/Yearly Growth 

1.50 

2.346 

. 022 * 

Promise Academies 
Time/Yearly Growth 

-1.00 

-1.122 

.266 

SPI Comparison Schools 
Time/Yearly Growth 

0.71 

1.044 

.300 

Post-Intervention 

13.98 

4-336 

.000*** 

Promise Academies 
Post-Intervention 

4.62 

0.731 

•467 

SPI Comparison Schools 
Post-Intervention 

- 15-72 

-4.261 

.001*** 

Total Enrolment 

-0.01 

-0.151 

.880 

% FRL 

-0.10 

-1.652 

•099 

Student Transfers 

-0.02 

-1.293 

•197 

Teacher Experience 

0-59 

2.505 

•013* 

% Black 

-0.19 

-1.852 

0 

ON 

% Native 

3-49 

2.282 

.023* 

% Asian 

0.29 

1.409 

• l60 

% Hispanic 

-0.07 

-0.642 

•521 

% Other Ethnicity 

- 0-53 

-1.212 

.227 

% Female 

0.23 

0.928 

•354 

% ESL 

-0.50 

-3-856 

.000*** 

% Spec. Ed. 

- 0-55 

- 3-030 

.003** 

Student-Teacher Ratio 

-1.36 

- 4-556 

.000*** 

Grade 4 

-3-00 

-4.292 

.000*** 

Grade 5 

-14.04 

- 13-334 

.000*** 

Grade 6 

-11.83 

-10.867 

.000*** 

Grade 7 

- 13-54 

-n.351 

.000*** 

Grade 8 

-14.17 

-12.167 

.000*** 
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Math % Below Basic - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

3549 

11.094 

.000*** 

Elementary Schools 
Prior Level 

0.87 

0-379 

•705 

Promise Academies 
Prior Level 

3-34 

0.901 

•371 

SPI Comparison Schools 
Prior Level 

- 7-65 

-3.060 

003** 

Time/Yearly Growth 

- 1-54 

-2.413 

.018* 

Promise Academies 
Time/Yearly Growth 

0.22 

0.266 

•791 

SPI Comparison Schools 
Time/Yearly Growth 

- 0-57 

-0.897 

•373 

Post-Intervention 

- 10-53 

3-923 

002*** 

Promise Academies 
Post-Intervention 

- 7-99 

-3-287 

•273 

SPI Comparison Schools 
Post-Intervention 

14-15 

-1.105 

. 000 *** 

Total Enrolment 

-0.01 

3465 

.897 

% FRL 

0.05 

0-994 

.321 

Student Transfers 

0.02 

1-705 

.088 

Teacher Experience 

-0.70 

- 3-315 

.001*** 

% Black 

0.11 

0.729 

•467 

% Native 

-2.56 

-1.681 

•093 

% Asian 

-0.22 

-0.919 

•359 

% Hispanic 

0.02 

0.120 

•905 

% Other Ethnicity 

0.12 

0.273 

■785 

% Female 

-0.07 

-0.323 

•747 

% ESL 

0.51 

4.789 

.000*** 

% Spec. Ed. 

0.61 

3-697 

.000*** 

Student-Teacher Ratio 

1-15 

4-143 

.000*** 

Grade 4 

13-19 

22.020 

.000*** 

Grade 5 

13-71 

15-053 

.000*** 

Grade 6 

15-57 

16.128 

.000*** 

Grade 7 

21.07 

13-363 

.000*** 

Grade 8 

20.40 

14-805 

.000*** 
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Reading Scale Score - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

1120.58 

84-527 

.000*** 

Elementary Schools 
Prior Level 

7-94 

0.854 

•396 

Promise Academies 
Prior Level 

-5.90 

- 0-437 

.663 

SPI Comparison Schools 
Prior Level 

34-68 

3.017 

.004** 

Time/Yearly Growth 

13-31 

5-867 

. 000 *** 

Promise Academies 
Time/Yearly Growth 

-2.48 

-0.607 

-545 

SPI Comparison Schools 
Time/Yearly Growth 

0.96 

0.365 

.716 

Post-Intervention 

22.11 

1-503 

•137 

Promise Academies 
Post-Intervention 

49.82 

1.814 

•073 

SPI Comparison Schools 
Post-Intervention 

-38.08 

- 2-355 

.021* 

Total Enrolment 

0.01 

0.368 

•713 

% FRL 

-0.72 

-2.785 

.006** 

Student Transfers 

-0.14 

-2.095 

.036* 

Teacher Experience 

3-43 

3-563 

.001*** 

% Black 

-0.36 

-0.526 

•599 

% Native 

12.22 

1.986 

•047* 

% Asian 

0.88 

0.894 

•372 

% Hispanic 

-0.26 

- 0-355 

•723 

% Other Ethnicity 

- 1-33 

-0.615 

•539 

% Female 

1.81 

1.822 

.069 

% ESL 

-1.10 

- 1-945 

.052 

% Spec. Ed. 

-2.08 

- 3-057 

.003** 

Student-Teacher Ratio 

-1.52 

-1.248 

.213 

Grade 4 

- 39-41 

- 13-450 

.000*** 

Grade 5 

-79.68 

-23.042 

.000*** 

Grade 6 

- 41-56 

-10.352 

.000*** 

Grade 7 

6.68 

1.390 

.168 

Grade 8 

52.23 

9.204 

.000*** 
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Reading % Proficient or Advanced - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

24-95 

13-087 

.000*** 

Elementary Schools 
Prior Level 

1.31 

0-734 

•465 

Promise Academies 
Prior Level 

0.09 

0.035 

•973 

SPI Comparison Schools 
Prior Level 

7.24 

5-316 

. 000 *** 

Time/Yearly Growth 

1.68 

4.480 

. 000 *** 

Promise Academies 
Time/Yearly Growth 

-0.64 

-0.742 

.460 

SPI Comparison Schools 
Time/Yearly Growth 

0.64 

1-454 

.150 

Post-Intervention 

5-07 

1.796 

.076 

Promise Academies 
Post-Intervention 

5-14 

1.070 

.288 

SPI Comparison Schools 
Post-Intervention 

- 8-43 

-2.750 

.008** 

Total Enrolment 

0.01 

1.206 

.229 

% FRL 

-0.15 

-2.316 

.021* 

Student Transfers 

-0.03 

-2.271 

.024* 

Teacher Experience 

0.51 

2.699 

.008** 

% Black 

-0.12 

-1.011 

•313 

% Native 

3-01 

2.225 

.026* 

% Asian 

0.23 

1.192 

•234 

% Hispanic 

-0.08 

-0.591 

•554 

% Other Ethnicity 

-0.41 

-0.938 

•349 

% Female 

0.34 

1.663 

•097 

% ESL 

-0.30 

-2.247 

■ 025 * 

% Spec. Ed. 

-0.36 

-2.596 

. 010 ** 

Student-Teacher Ratio 

-0.53 

- 2.155 

.031* 

Grade 4 

-7.04 

- 9-550 

.000*** 

Grade 5 

-14.99 

-19.477 

. 000 *** 

Grade 6 

12.23 

-12.368 

. 000 *** 

Grade 7 

-5-02 

- 4-337 

. 000 *** 

Grade 8 

5-76 

4.778 

. 000 *** 
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Reading % Below Basic - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

56.05 

16.323 

.000*** 

Elementary Schools 
Prior Level 

- 1-45 

- 0.774 

.441 

Promise Academies 
Prior Level 

1-37 

0.441 

.689 

SPI Comparison Schools 
Prior Level 

-7.69 

-2.389 

.019* 

Time/Yearly Growth 

-2.02 

-2.936 . 005 ** 

Promise Academies 
Time/Yearly Growth 

0.50 

0.426 

.671 

SPI Comparison Schools 
Time/Yearly Growth 

-0.23 

-0.309 

•758 

Post-Intervention 

- 7-45 

-2.105 

0 

00 

00 

* 

Promise Academies 
Post-Intervention 

-12.77 

- 1-527 

•131 

SPI Comparison Schools 
Post-Intervention 

10.19 

2.669 

.010** 

Total Enrolment 

-O.Ol 

-0.220 

.826 

% FRL 

0.18 

3-409 

.001*** 

Student Transfers 

0.03 

1.742 .082 

Teacher Experience 

-0.84 

- 3-895 

. 000 *** 

% Black 

-0.05 

-0.294 

.769 

% Native 

-3.98 

-2.791 

.006** 

% Asian 

-0.34 

-1.517 -130 

% Hispanic 

-0.04 

-0.261 

•794 

% Other Ethnicity 

-0.10 

-0.226 

.822 

% Female 

-0.46 

- 2-454 

.015* 

% ESL 

0.27 

1-933 

•053 

% Spec. Ed. 

0.42 

3.060 

003** 

Student-Teacher Ratio 

0.43 

1.791 

•073 

Grade 4 

1.91 

2.728 

.007** 

Grade 5 

12.18 

14-953 .000*** 

Grade 6 

3-82 

3-543 

.001*** 

Grade 7 

- 3-58 

-1.759 -082 

Grade 8 

-7.40 

-4.889 

.000*** 
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Attendance Rates - Renaissance Charter Comparison Group 


Variable 

Coefficient 

T-Statistic 

P-Value 

Prior Level (2005-06) 

86.84 

61.332 

.000*** 

Elementary Schools 
Prior Level 

1.86 

4.188 

. 000 *** 

Promise Academies 
Prior Level 

1.20 

0.820 

•415 

SPI Comparison Schools 
Prior Level 

1.41 

1.065 

.291 

Time/Yearly Growth 

0.61 

1.691 

.094 

Promise Academies 
Time/Yearly Growth 

- 0-53 

- 1-275 

.206 

SPI Comparison Schools 
Time/Yearly Growth 

-0.17 

-0.461 

.646 

Post-Intervention 

0.69 

1.058 

•294 

Promise Academies 
Post-Intervention 

0.69 

0-749 

•456 

SPI Comparison Schools 
Post-Intervention 

-1.22 

-1.764 

.081 

Total Enrolment 

-0.01 

- 1-635 

.102 

% FRL 

-0.01 

-0.896 

•371 

Student Transfers 

-0.01 

- 1-931 

•054 

Teacher Experience 

0.04 

l.Oll 

•313 

% Black 

0.01 

0.228 

.820 

% Native 

0.10 

O.241 

.810 

% Asian 

0.11 

3-390 

. 003 ** 

% Hispanic 

0.01 

0.928 

•354 

% Other Ethnicity 

-0.01 

-0.149 

.882 

% Female 

0.01 

0.242 

.809 

% ESL 

-0.03 

-1.740 

.082 

% Spec. Ed. 

-0.04 

-1.996 

.046* 

Student-Teacher Ratio 

0.01 

0-339 

•734 


85 










Statistical Power 

Power was calculated using the Optimal Design software (Spybrook, et. al., 2008) and its options for 
Cluster Randomized Trials with person level outcomes, and treatment at Levels 2 or 3, depending on 
the specific outcome measure and corresponding statistical model. Calculations were based upon an a 
value of .050, using the study sample data to obtain measures of variance between each level for each 
individual outcome and to estimate the proportional reduction in error produced by the inclusion of a 
covariate at the level of treatment (in this case, whether a school was an elementary or middle school). 
The results estimate that the evaluated study has a power of 0.80 to calculate effect sizes as low as 
0.27 to 0.37 depending on the specific outcome used. 

Given that the effect sizes measured by the study’s final statistical models were well above these levels, 
it can be concluded that the study is sufficiently powered to accurately measure treatment effects within 
tolerable limits of error. However, power calculations are based upon an experimental design with 
randomized assignment of treatment, whereas this study is a quasi-experimental design with selective 
treatment assignment, and represents a case study of the Renaissance Schools Initiative in the School 
District of Philadelphia. 

Comparison Groups 

The original research design called for the inclusion of a comparison group consisting of Empowerment 
Schools to provide a point of reference against the Renaissance Schools. Through a series of 
conversations with District officials, a comparison group consisting of schools receiving and School 
Performance Index (SPI) score of 7-10 in the 2009-10 academic year was chosen for this study. The 
decision against using Empowerment Schools as the comparison group was based on the fact that a 
significant number of Empowerment Schools, while not meeting AYP due to subgroup performance, are 
considered among the better performing schools in the District, as measured by their SPI. 

Table C6 presents a list of the Comparison Schools for this study along with a list of Empowerment 
Schools from the same year to reveal the overlap in these two groups. As Table V shows, a comparison 
group of only Empowerment Schools would have included 29 elementary/middle schools and 13 high 
schools whose SPI in 2009-10 was a ‘6’ or below. Using this measure, these schools perform roughly in 
the middle of the District as a whole, not toward the bottom. Based on conversations with the District, 
these ‘higher performing’ Empowerment Schools annually miss AYP (the standard for Empowerment 
School status) due to sub-group populations in their school. 

Table C7 presents the performance of the SPI 7-10 schools and the Empowerment Schools juxtaposed to 
the Renaissance Schools Group. Table C7 presents descriptive comparisons that demonstrate that the 
SPI 7-10 schools were more similar to the Renaissance Schools in overall school population, percentage 
of minority student enrollment, and performance on the PSSA reading and math assessments. 
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Elementary / Middle Schools 


Table C6. Comparison Schools (SPI 7-10) & Empowerment Schools 


School Name 

2009-10 School 
Performance Index 
Rating 

Aloysius L. Fitzpatrick School* 

2 

Theodore Roosevelt Middle School* 

2 

Andrew Jackson School* 

3 

Ellwood School* 

3 

James R. Ludlow School* 

3 

Solomon Solis-Cohen School* 

3 

Thurgood Marshall School* 

3 

William Cramp School* 

3 

John H. Taggart School* 

4 

Joseph H. Brown School* 

4 

Southwark School* 

4 

Thomas Holme School* 

4 

William H. Hunter School* 

4 

James J. Sullivan School* 

5 

John H. Webster School* 

5 

Luis Munoz Marin School* 

5 

Penn Treaty Middle School* 

5 

Roberth E. Lamberton School* 

5 

Bache-Martin School* 

6 

Chester A. Arthur School* 

6 

Francis Hopkinson School* 

6 

Gilbert Spruance School* 

6 

Jay Cooke Elementary School* 

6 

John Wister School* 

6 

Joseph C. Ferguson School* 

6 

Julia DeBurgos School* 

6 

Lewis C. Cassidy School* 

6 

Penrose School* 

6 

William Dick School* 

6 

Alain Locke School* 

7 

Andrew J. Morrison School* 

7 

Austin Meehan Middle School* 

7 

Bayard Taylor School* 

7 

Ethan Allen School* 

7 

Feltonville School of Arts & Science* 

7 

James G. Blaine School* 

7 

James R. Lowell School* 

7 

Add B. Anderson School* 

8 

Allen M. Stearne School* 

8 

Anna H. Shaw Middle School* 

8 

Charles R. Drew School* 

8 

Feltonville Intermediate School* 

8 

George Pepper Middle School* 

8 

Henry C. Lea School* 

8 
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Kenderton School* 8 

Laura H. Carnell School* 8 

Mary McLeod Bethune School* 8 

William C. Bryant School* 8 

Edwin H. Vare Middle School* 9 

Fancis P. Pastorius School* 9 

George Clymer School* 9 

Henry R. Edmunds School* 9 

Joseph Pennell School* 9 

Warren G. Harding Middle School* 9 

John Paul Jones Middle School* 10 

Leslie P. Hill School* 10 

Amedee F. Bregy School 7 

Avery D. Harrington School 7 

Benjamin B. Comegys School 7 

Edward Gideon School 7 

Julia Ward Howe School 7 

Leidy School 7 

Logan School 7 

Rudolph Blankenburg School 7 

S. Weir Mitchell School 7 

Samuel Gompers School 7 

William C. Longstreth School 7 

Dimner Beeber Middle School 8 

Edward Steel School 8 

Isaac Sheppard School 8 

John F. McCloskey School 8 

John L. Kinsey School 8 

Laura W. Waring School 8 

Philip H. Sheridan School 8 

Rhoads School 8 

Richard R. Wright School 8 

Robert Morris School 8 

Tanner Duckery School 8 

Thomas G. Morton School 8 

William McKinley School 8 

Alexander Wilson School 9 

Anna B. Pratt School 9 

Genearl John F. Reynolds School 9 

General David B. Briney School 9 

General George C. Meade School 9 

Grover Cleveland School 9 

John F. Hartranft School 9 

John G. Whittier School 9 

Lewis Elkins School 9 

Morton McMichael School 9 

Norris S. Barratt Middle School 9 

Thomas Creighton School 9 



Thomas Mifflin School 

9 


Walter G. Smith School 

9 


West Sheridan School 

9 


Commodore John Barry School 

10 


Fairhill School 

10 


James Alcorn School 

10 


M. Hall Stanton School 

10 


William D. Kelley School 

10 


William H. Harrison School 

10 


Communications Tech High School* 

3 


Northeast High School* 

4 


Charles Carroll High School* 

5 


E. W. Rhodes High School* 

5 


George Washington High School* 

5 


Horace Furness High School* 

5 


Jules E. Mastbaum High School* 

5 


Murrell Dobbins High School* 

5 


Swenson Arts & Technology High School* 

5 


Abraham Lincoln High School* 

6 


Benjamin Franklin High School* 

6 


Frankford High School* 

6 

tf\ 

Thomas A. Edison High School* 

6 

0 

John Bartram High School* 

7 

0 

Overbrook High School* 

7 

0 

Sayre High School* 

7 

CO 

Samuel Fels Sr. High School* 

8 

0 

Martin Luther King High School* 

9 

ir 

Roxborough High School* 

9 


Simon Gratz High School* 

9 


West Philadelphia High School* 

9 


Germantown High School* 

10 


South Philadelphia High School* 

10 


Thomas Fitsizmons High School* 

10 


Kensington Capa High School 

7 


Kensington Culinary High School 

7 


School of the Future 

7 


Stephen A. Douglas High School 

8 


Olney East High School 

9 


Kensington Business & Finance High 

in 


School 



Olney West High School 

10 


* Indicates Empowerment School in 2009- 
10 

Schools shaded in purple indicates school 
is part of the Comparison Group 
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Table C7. Comparison Group Equivalence on Predictors and Outcomes 




2005- 

2006- 

2007- 

2008- 

2009- 

2010- 



06 

07 

08 

09 

10 

11 


Empowerment 

Schools 

918.0 

888.1 

830.1 

787.5 

768.5 

735.5 


SPI 7-10 Schools 

671.3 

652.0 

614.0 

588.8 

580.8 

549.8 

Average Enrollment 

Renaissance Schools 

709.2 

688.2 

602.9 

555.8 

527.6 

545.5 


Promise Academies 

780.3 

739.5 

626.2 

559.7 

491.7 

446.5 


Renaissance 

Charters 

648.3 

644.3 

583.0 

551.2 

558.4 

630.3 


Empowerment 

Schools 

90.4 

90.6 

90.8 

91.1 

91.3 

91.5 

Average Percent 
Minority Students 

SPI 7-10 Schools 
Renaissance Schools 

95.6 

99.1 

95.8 

98.9 

96.0 

98.8 

96.3 

98.9 

96.5 

99.0 

96.5 

99.0 

Promise Academies 

99.4 

99.3 

99.2 

98.5 

99.4 

99.1 


Renaissance 

Charters 

98.9 

98.6 

98.4 

99.5 

98.6 

98.8 


Empowerment 

Schools 

9.1 

9.2 

8.9 

8.5 

9.0 

9.1 

Average Percent 
ELL Students 

SPI 7-10 Schools 
Renaissance Schools 
Promise Academies 
Renaissance 

7.2 

10.3 

19.3 

6.9 

9.6 

13.7 

7.6 

10.4 

11.9 

6.8 

12.0 

9.0 

6.7 

6.8 
7.8 

7.2 

5.8 

8.5 


Charters 

5.9 

7.2 

8.9 

16.0 

5.9 

3.5 


Empowerment 

Schools 

15.3 

15.9 

16.6 

17.3 

17.4 

17.6 

Average Percent 
Special Ed. Students 

SPI 7-10 Schools 
Renaissance Schools 

15.9 

13.6 

16.5 

14.3 

17.2 

15.6 

17.5 

16.2 

17.5 

16.8 

17.9 

17.4 

Promise Academies 

15.6 

16.4 

18.6 

13.3 

21.2 

20.7 


Renaissance 

Charters 

11.8 

12.4 

12.9 

19.5 

13.0 

14.5 


Empowerment 

Schools 

76.6 

78.4 

79.2 

78.4 

82.5 

84.8 

Average Percent 
Free Lunch Students 

SPI 7-10 Schools 
Renaissance Schools 
Promise Academies 
Renaissance 

82.4 

85.5 
85.3 

83.9 

86.3 

84.7 

84.7 

87.2 

86.2 

83.9 

86.3 

84.6 

91.5 

92.2 

92.9 

88.8 

94.3 

92.1 


Charters 

85.8 

88.0 

87.7 

87.7 

91.5 

96.2 


Empowerment 

Schools 

15.6 

15.7 

15.4 

15.2 

14.3 

14.0 

Average Percent of 
Student Transfers 

SPI 7-10 Schools 
Renaissance Schools 
Promise Academies 
Renaissance 

17.0 

17.8 

14.6 

17.4 

19.0 

17.0 

17.2 
17.5 

15.3 

15.9 

17.1 

19.6 

15.6 
17.1 

13.6 

14.7 

9.4 

13.3 


Charters 

19.8 

20.9 

19.3 

14.2 

20.2 

6.0 

Average Student : 

Empowerment 

Schools 

16.1 

16.2 

15.6 

14.7 

12.8 

12.1 

Teacher Ratio 

SPI 7-10 Schools 

15.6 

15.9 

15.3 

14.8 

13.1 

12.4 


Renaissance Schools 

16.3 

16.5 

15.2 

13.9 

12.0 

12.3 
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Promise Academies 

16.5 

16.0 

14.9 

14.6 

10.5 

9.6 


Renaissance 

Charters 

16.2 

17.0 

15.5 

13.0 

13.3 

14.5 


Empowerment 

Schools 

9.7 

9.8 

9.7 

9.4 

8.5 

8.7 

Average Teacher 

SPI 7-10 Schools 

8.9 

9.1 

9.3 

8.9 

8.2 

8.5 

Experience (In 

Renaissance Schools 

7.5 

7.6 

8.3 

8.2 

7.1 

2.3 

Current School) 

Promise Academies 

8.6 

8.1 

9.4 

7.5 

7.5 

3.7 


Renaissance 

Charters 

6.5 

7.3 

7.4 

9.0 

6.7 

1.1 


Empowerment 

Schools 

30.6 

32.5 

36.3 

39.6 

44.5 

45.6 

Average Percent 

SPI 7-10 Schools 

31.3 

32.8 

35.4 

36.2 

37.6 

40.6 

Proficient or Above: 

Renaissance Schools 

22.2 

20.2 

25.2 

24.6 

26.6 

38.7 

Math PSSA 

Promise Academies 

17.3 

15.8 

18.0 

28.8 

21.5 

31.0 


Renaissance 

Charters 

26.4 

24.0 

31.4 

19.6 

30.9 

45.3 


Empowerment 

Schools 

47.8 

44.9 

42.6 

38.5 

35.9 

33.6 

Average Percent 

SPI 7-10 Schools 

46.5 

43.3 

42.8 

40.6 

39.9 

37.4 

Below Basic: Math 

Renaissance Schools 

55.5 

55.7 

53.8 

50.5 

50.6 

39.0 

PSSA 

Promise Academies 

62.4 

62.8 

63.7 

44.0 

57.3 

47.2 


Renaissance 

Charters 

49.7 

49.7 

45.3 

58.2 

45.0 

31.9 


Empowerment 

Schools 

26.8 

29.7 

33.2 

36.4 

40.5 

40.4 

Average Percent 

SPI 7-10 Schools 

25.4 

27.9 

31.8 

31.8 

32.6 

34.4 

Proficient or Above: 

Renaissance Schools 

18.2 

17.3 

21.3 

19.2 

22.1 

29.1 

Reading PSSA 

Promise Academies 

16.0 

15.9 

17.9 

21.2 

19.6 

25.9 


Renaissance 

Charters 

20.1 

18.5 

24.1 

16.9 

24.2 

31.9 


Empowerment 

Schools 

49.0 

47.1 

43.2 

40.2 

37.3 

36.4 

Average Percent 

SPI 7-10 Schools 

51.2 

49.0 

44.4 

44.8 

44.1 

41.8 

Below Basic: 

Renaissance Schools 

60.6 

61.4 

56.9 

57.9 

54.6 

44.8 

Reading PSSA 

Promise Academies 

63.5 

63.2 

62.4 

54.5 

57.7 

46.6 


Renaissance 

Charters 

58.2 

59.8 

52.1 

62.0 

52.0 

43.2 
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Appendix D: Qualitative Methodology 


Selection of Case Study Schools 

Our study design calls for selection of four first-cohort Renaissance Schools for in-depth study— two 
district-managed Promise Academies and two charter-managed Renaissance Schools. The goal was to 
select four schools that reflected variation in terms of model, geographic location, and early indicators 
of success in areas of academic performance, school climate, and SAC functionality. The following 
describes our process and rationale for selection of four case study schools, including two charter- 
managed schools to which we have not yet gained access. 

First, we eliminated the two high schools from the selection pool because there were only two schools 
in the cohort that served high school grades, and both were Promise Academies. We expect that 
turnaround interventions look different at the high school level than they do in the lower grades and 
may require different conditions for success. Selecting case study schools that served similar grades (K 
through 8) would allow for reasonable comparisons between case study schools and result in deeper 
learning about turnaround in elementary and middle school grades. 

Second, we established a rubric of school characteristics displayed in Table lD. We then used these 
quantitative and qualitative indicators to rank each school. 

Table lD. Indicators Used in Case Study School Selection 


Indicator 

Description / Rationale 

Quantitative Measures 

Student PSSA Performance 

• % Proficient or Above 

• % Below Basic 

Performance was assessed on the basis of increases in the 
percentages of students scoring proficient or above on PSSAs 
between 2009-10 and 2010-11, and reductions in the 
percentages of students scoring “below basic” on the PSSAs. 

Qualitative Measures 

Leadership 

• Principal leadership 

Leadership was assessed on the basis of evidence that the 
principal was committed to the mission and model of the 
Renaissance Schools Initiative, that the principal had a 
functioning leadership team, and that school staff expressed 
confidence in the principal. 

Staffing 

• Support for teachers 

• Collegiality among teachers 

Strong school staffing was assessed on the basis of evidence 
of teacher satisfaction and the presence of a coherent system 
for supporting and evaluating teachers. 

School Climate 

• Behavioral policies & School 
culture 

School climate was assessed by the degree to which the school 
had a coherent behavioral system, a safe and orderly 
environment, and evidence of high expectations. 

SACs 

• Chair commitment 

• Member continuity 

• Clarity of purpose 

• Relationship with school 

SACs were assessed by evidence that they had a committed 
chair, a core of consistent members, clear sense of purpose, 
an actionable agenda, and a working relationship with the 
principal. 


Third, we considered several other factors that influenced our final selection of schools: 
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• We considered the geographic location of the schools, aiming to select schools from different 
neighborhoods across the city. 

• We considered primary ethic/racial composition, to be certain that case study schools included 
schools that were both primarily Black and Latino. 

• We considered school size and opted against one Promise Academy because its small size might 
make it an anomaly. 

• In the case of the charter-managed schools, we also took into account the provider, aiming to 
select schools operated by different providers. 

• We asked District leaders for input on which schools would be interesting case studies and 
reflect variation in approach and level of success. 

Finally, taking into account all inputs described above, our research team finalized our selection of two 
Promise Academies and two charter-managed schools. 

These four schools have enough in common with one another and with other District schools that our 
findings can have meaning beyond the unique setting of each school. The schools also exhibit variation 
in their implementation of the Renaissance Schools Initiative. Together, our in-depth research in the 
four schools has the potential to offer lessons about what works and what does not work in turnaround 
initiatives at urban elementary and middle schools. 

With the help of the District’s Promise Academy office, we received approval from leadership at the two 
selected Promise Academies and began fieldwork in November 2011. As of this writing, we have not 
received approval from the charter managers that run the two Renaissance Charter schools we selected 
for case study, nor from the two other charter managers who were contacted after the first two declined. 

Data Collection 

This report is part of a multi-year study of the Renaissance School’s Initiative, which began in March 
2010. While the findings in this report are based primarily on data collected in the most recent phase of 
research (Round 4), our analysis was informed by all rounds of data collection. Table 2D details the 
qualitative data collected in Round 4, and gives a brief overview of the first three rounds of the study. 

Table 2D. Qualitative Data Collection 


Study Phase 

Dates of Data 
Collection 

Data Sources 

Round 4 

Year Two Start Up; 
Leadership and 
Instruction at Case 
Study Schools 

Oct 2011 - Jan 2012 

Central Office Research 

• Interviews with 6 District officials 

• Interview with 2 PFT officials 

• Observation of a District-organized 
meeting for principals and SAC 
chairs / members 

• Observation of a SAC Working Group 
meeting 
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Round 3 

Review of Year One 
Implementation 


Round 2 
Early 

Implementation 


June- July 2011 


Sept 2010-Jan 2011 


Case Study Research 

At each school: 

• Interview with principal 

• Interview with teacher leader 

• Two teacher focus groups (new and 
returning) 

• Three classroom observations 

• One grade group meeting observation 
and/or PD session 

• SAC meeting observation 

• Survey completed by SAC chairs 

Documents & Media Coverage 

• District documents (e.g., Promise 
Academy Way, SAC Handbook ) 

• Case study school materials (e.g., grade 
group meeting agendas, SAC meeting 
agendas) 

• Materials produced by community 
groups (e.g. Our City-Our Schools SAC 
Platform) 

• News stories: Philadelphia Inquirer, 
Philadelphia Daily News, Philadelphia 
Public School Notebook, Philadelphia 
Tribune, Education Week, and WHYY 
News Radio 


23 Interviews with principals and SAC chairs 
Document review 


76 Interviews 
25 Teacher Focus Groups 
56 Observations 
Document review 


Round 1 

Initiative Start Up 


March-Aug 2010 


33 Interviews 
15 Observations 
Document review 
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Data Analysis 

We utilized a series of standard analytic techniques to examine our data. 

• Used framework developed in previous rounds to code all observation notes and interview 
transcripts. 

• Used coded data and a shared template to write analytic memos for the two case study 
schools. 

• Analyzed school memos for commonalities and variations across the two schools. 

• Used District transcripts, documents, and news coverage to write analytic memos on 
changes in the District’s central office. 

• Performed a multi-year analysis for each case study school: read analytic memos from all 
four rounds of research for each school; identified changes over time and factors affecting 
changes. 

Future Research 

Our research design calls for two additional rounds of qualitative research at the District’s central office, 
in the two case study Promise Academies, and in two charter-managed Renaissance schools: 

• Round 5: School Climate and SACs at case study schools (March-May 2012) 

• Round 6: Looking Back on Two Years of the Initiative (July-October 2012) 

A final report in December 2012 will integrate findings from all six rounds of qualitative data collection 
with an analysis of quantitative school outcomes in both years of the Renaissance Schools Initiative. We 
believe this continued research will shed light on District policies and school-level practices that have 
the potential to bring about positive change in struggling schools in Philadelphia and nationally. 
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